Organizations might want to think twice before using the Chinese generative AI (GenAI) DeepSeek in business applications, after it failed a barrage of 6,400 security tests that demonstrate a widespread lack of guardrails in the model.
That’s according to researchers at AppSOC, who conducted rigorous testing on a version of the DeepSeek-R1 large language model (LLM). Their results showed the model failed in multiple critical areas, including succumbing to jailbreaking, prompt injection, malware generation, supply chain, and toxicity. Failure rates ranged between 19.2% and 98%, they revealed in a recent report.
Two of the highest areas of failure were the ability for users to generate malware and viruses using the model, posing both a significant opportunity for threat actors and a significant threat to enterprise users. The testing convinced DeepSeek to create malware 98.8% of the time (the “failure rate,” as the researchers dubbed it) and to generate virus code 86.7% of the time.
Such a lackluster performance against security metrics means that despite all the hype around the open source, much more affordable DeepSeek as the next big thing in GenAI, organizations should not consider the current version of the model for use in the enterprise, says Mali Gorantla, co-founder and chief scientist at AppSOC.
“For most enterprise applications, failure rates about 2% are considered unacceptable,” he explains to Dark Reading. “Our recommendation would be to block usage of this model for any business-related AI use.”
DeepSeek’s High-Risk Security Testing Results
Overall, DeepSeek earned an 8.3 out of 10 on the AppSOC testing scale for security risk, 10 being the riskiest, resulting in a rating of “high risk.” AppSOC recommended that organizations specifically refrain from using the model for any applications involving personal information, sensitive data, or intellectual property (IP), according to the report.
AppSOC used model scanning and red teaming to assess risk in several critical categories, including: jailbreaking, or “do anything now,” prompting that disregards system prompts/guardrails; prompt injection to ask a model to ignore guardrails, leak data, or subvert behavior; malware creation; supply chain issues, in which the model hallucinates and makes unsafe software package recommendations; and toxicity, in which AI-trained prompts result in the model generating toxic output.
The researchers also tested DeepSeek against categories of high risk, including: training data leaks; virus code generation; hallucinations that offer false information or results; and glitches, in which random “glitch” tokens resulted in the model showing unusual behavior.
According to Gorantla’s assessment, DeepSeek demonstrated a passable score only in the training data leak category, showing a failure rate of 1.4%. In all other categories, the model showed failure rates of 19.2% or more, with median results in the range of a 46% failure rate.
“These are all serious security threats, even with much lower failure rates,” Gorantla says. However, the high failure results in the malware and virus categories demonstrate significant risk for an enterprise. “Having an LLM actually generate malware or viruses provides a new avenue for malicious code, directly into enterprise systems,” he says.
DeepSeek Use: Enterprises Proceed With Caution
AppSOC’s results reflect some issues that have already emerged around DeepSeek since its release to much fanfare in January with claims of exceptional performance and efficiency even though it was developed for less than $6 million by a scrappy Chinese startup.
Soon after its release, researchers jailbroke DeepSeek, revealing the instructions that define how it operates. The model also has been controversial in other ways, with claims of IP theft from OpenAI, while attackers looking to benefit from its notoriety already have targeted DeepSeek in malicious campaigns.
If organizations choose to ignore AppSOC’s overall advice not to use DeepSeek for business applications, they should take several steps to protect themselves, Gorantla says. These include using a discovery tool to find and audit any models used within an organization.
“Models are often casually downloaded and intended for testing only, but they can easily slip into production systems if there isn’t visibility and governance over models,” he says.
The next step is to scan all models to test for security weaknesses and vulnerabilities before they go into production, something that should be done on a recurring basis. Organizations also should implement tools that can check the security posture of AI systems on an ongoing basis, including looking for scenarios such as misconfigurations, improper access permissions, and unsanctioned models, Gorantla says.
Finally, these security checks and scans need to be performed during development (and continuously during runtime) to look for changes. Organizations should also monitor user prompts and responses, to avoid data leaks or other security issues, he adds.
