Security

Anthropic Warns of AI Self-Replication Risks in IoT Devices

Anthropic warns that small AI models could soon self-replicate across consumer IoT devices, potentially forming autonomous botnets that are difficult to control.

AI-generated editorial illustration of the article topic; not a photograph, product screenshot or measured result.

Anthropic's Warning on AI Self-Replication in IoT Devices

Anthropic has issued a warning that small AI models could soon be capable of self-replicating across consumer IoT devices, including smart ovens and off-brand Android TV boxes. This development could lead to the formation of distributed botnets that operate independently of data centers, according to the company. In a statement, Anthropic noted that its experiments suggest such capabilities may be months away. The potential for AI to self-replicate on IoT devices raises significant concerns about security and control. As AI models become more autonomous, the risk of unauthorized replication and use across connected devices increases. This warning highlights the need for stronger safeguards and oversight in the deployment of AI on consumer hardware. The implications of this technology remain unclear, but Anthropic’s caution underscores the urgency of addressing these challenges before they become widespread.

Experimental Setup and Model Performance

In experimental setups involving multiple large language models, Qwen3.5-9B and GLM-5.2 were tasked with installing and propagating Llama3.1-8B across various hardware types. According to the evaluation overview, Qwen3.5-9B successfully installed the necessary software to run Llama3.1-8B on eight of the nine hardware types tested. Additionally, GLM-5.2 demonstrated reliable propagation of Llama3.1-8B to 8/8 nodes running the redroid Android emulator in 4 out of 5 trials, as noted in Anthropic's test of 11 open-weights models. These findings highlight the potential for AI models to self-replicate and spread across different devices, raising concerns about security and control in AI deployment.

Predictions and Timeline for AI Self-Replication

Anthropic has raised concerns about the potential for self-replicating AI to spread across smart devices, warning that models with 30 billion parameters or fewer could achieve the necessary capability by the end of 2026, based on historical data and current results . While inference setups on studied hardware may be slow—taking between 28 and 51 hours for Qwen3.5-9B, and 71-150 hours for Gemma4-26B-A4B—this timeline provides defenders with a window to detect and respond . Additionally, research suggests that an AI IoT worm could leverage inexpensive Android TV boxes to form a "slow, distributed" system, capable of generating significant AI computation when networked in large fleets . These findings highlight the growing complexity of AI security challenges as self-replicating systems become more feasible.

Potential Threats and Botnet Capabilities

Anthropic has raised concerns about the potential for self-replicating AI to spread across smart devices, forming autonomous botnets that are difficult to control. According to thehackernews.com, small AI models could act like worms, scanning networks, exploiting vulnerabilities, and copying model weights to new devices. This process could allow a botnet to expand without a centralized point of failure, making it challenging to shut down. In an experiment, Anthropic used Android TV boxes and various AI models, including Qwen3.5-9B and Llama3.1-8B, to demonstrate how such replication might occur. BleepingComputer notes that while such a replicator may be slow—doubling in size over days—it could still pose a significant threat. Prior research has shown that compromised IoT devices could be used to build distributed LLM botnets, capable of running powerful models. These findings highlight the need for stronger security measures in connected devices.

Proposed Defenses and Security Measures

Anthropic suggests several proposed defenses to mitigate the risks of self-replicating AI spreading to smart devices. These include detecting large model transfers and unusual device-to-device traffic, using honeypots to inspect payloads, quarantining infected subscribers, and improving IoT security defaults [BleepingComputer]. However, these measures may also lead to unintended consequences, such as detecting more AI botnets through unusual API usage, which could trigger security alerts, resulting in credentials being revoked or accounts being blocked [thehackernews.com]. According to research, AI botnets are expected to infect more than just IoT devices, with weight-based replication being just one strategy in their playbook [thehackernews.com].