DeepSeek Leads Industrial-Scale Distillation Attacks on AI Models, Researchers Find

The industrial‑scale distillation campaign uncovered by Anthropic’s security team underscores a new threat vector for foundation‑model providers. According to the internal report released by Anthropic, more than 24,000 fraudulent accounts were operated by three Chinese‑backed labs—DeepSeek, Moonshot AI and MiniMax—to harvest over 16 million conversational exchanges with Claude, the company’s flagship LLM. The attackers used the harvested data to “extract its capabilities” and accelerate the training of their own models, a process known as model distillation that effectively clones a proprietary system without direct access to its weights (Anthropic report). By automating account creation and prompting at scale, the groups turned what had previously been a niche research exercise into a quasi‑industrial operation, raising the stakes for any organization that monetizes API access.

The scale of the operation is notable not just for its volume but for its geopolitical implications. Ars Technica’s coverage of China’s rapid progress in “reasoning” AI notes that DeepSeek’s recent releases have begun to rival the performance of leading U.S. models, suggesting that the stolen Claude data could be a catalyst for closing the gap (Ars Technica). The report points out that DeepSeek, Moonshot AI and MiniMax have all positioned themselves as “open‑source‑friendly” challengers, yet the reliance on illicitly sourced data blurs the line between open collaboration and intellectual‑property theft. This dynamic mirrors earlier concerns about data‑poaching in the AI arms race, where state‑backed entities leverage aggressive data‑collection tactics to leapfrog commercial rivals.

From a business‑model perspective, the findings raise urgent questions about the sustainability of API‑driven revenue streams. Anthropic’s own figures—16 million Claude exchanges extracted via fraudulent accounts—represent a non‑trivial portion of the company’s usage volume, potentially translating into millions of dollars of lost revenue. VentureBeat’s recent piece on Nvidia’s memory‑sparsification technique highlights how reducing inference costs can make large‑scale model training more accessible, thereby lowering the barrier for actors who might otherwise be deterred by compute expenses (VentureBeat). When the cost of reproducing a model drops, the incentive to acquire it through illicit means grows, amplifying the risk to firms that rely on pay‑per‑call pricing.

Regulatory and defensive responses are still nascent. The Register notes that China’s DeepSeek has already launched a “free challenger” to OpenAI’s upcoming o1 model, indicating a willingness to compete aggressively in the high‑performance reasoning space (The Register). However, the report does not detail any coordinated industry effort to mitigate large‑scale distillation attacks, leaving individual providers to shoulder the burden of detection and prevention. Anthropic’s disclosure suggests that internal monitoring tools were finally able to correlate anomalous account behavior with the volume of Claude interactions, but the precise technical safeguards remain undisclosed.

The broader market implication is a potential recalibration of risk assessments for AI investors and corporate users. If model‑theft pipelines can be industrialized, the value proposition of proprietary LLMs may shift from pure performance to the robustness of their security ecosystems. Stakeholders will likely demand stronger authentication, usage‑pattern analytics, and perhaps legal frameworks that address cross‑border data‑extraction. Until such measures coalesce, the industry may see a wave of “shadow” models—replicas built on stolen data—competing in the same commercial arenas as their legitimate counterparts, eroding the competitive moat that companies like Anthropic have built around their flagship systems.