Category: Uncategorized

Ravie LakshmananOct 13, 2026Artificial Intelligence / IoT Security

Small artificial intelligence (AI) models may be months away from developing capabilities to self-replicate on IoT devices like smart ovens and off-brand Android TV boxes, according to a warning from Anthropic.

The frontier AI lab said small AI models could form autonomous, self-propagating worm botnets that are “challenging to shut down,” since “there’s no centralized point to shut off.”

Prior work has demonstrated that AI agents can act like one by scanning their network, executing an exploit, and copying model weights to a target machine across a GPU cluster with an aim to generate tailored attack strategies and use compromised machines as a botnet to run large language models (LLMs) in order to sustain its reasoning and expand in scale.

“Such an ‘AI worm’ could carry its model weights with it as it reproduces (or re-download them from a public hub), and then run inference directly on compromised hardware,” Anthropic said. “Effectively, the weights would become part of the malware payload.”

The idea, at a high level, explains how AI models could potentially break into internet-connected devices and use their computing power to run themselves and spread to other devices, without having to rely on expensive data centers all the time. A more powerful model that’s too large to fit into a single device could be split into small chunks and run across multiple machines to collectively run it.

A threat actor could build a distributed LLM botnet from poorly secured IoT devices, making it powerful enough to run a capable model. Some of the ways an AI worm could propagate include –

• Small models that replicate through a compromise of host machines on which they run, also called weight-based replicators. IoT devices make for a lucrative target because of their minimal security controls.
• Target datacenters and other high-compute targets to set up unmonitored rogue deployments, also classified as frontier replicators or open-weight replicators based on the model used.
• Steal an AI service provider’s API keys or platform credentials to run copies of itself, also called API-based replicators. In this case, the provider could be deceived into thinking it is serving ordinary API requests from a legitimate customer, without realizing that a malicious or self-replicating system is behind them. However, they are also likely to detect more as unusual API usage could trigger security alerts, leading to the credentials being revoked or the accounts getting blocked.

A fleet of 15 Smart TV boxes running distributed inference of Gemma4-26B-A4B.

In its test of 11 open-weights models from Alibaba, DeepSeek, Google, Meta MoonShot AI, OpenAI, and Zhipu AI (aka Z.ai), Anthropic said it found GLM-5.2 to be the only model to propagate Llama3.1-8B reliably to 8/8 nodes running the redroid Android emulator in 4 out of 5 trials.

It also revealed that an AI IoT worm powered by inexpensive Android TV boxes could be a “slow, distributed” system that can produce a meaningful amount of AI computation when a fleet comprising hundreds of thousands of such devices is networked together to run an inference engine.

Alternatively, it could take the form of a potent cyber model capable of breaching data centers at a high pace, or a model that scams people for compute credits.

“We don’t yet have a full rogue replicator, but agents are becoming increasingly capable of completing the individual steps in the replication chain (and passing on their adaptations to future copies of themselves),” Anthropic said.

“We expect AI botnets to infect more than IoT devices, and to use weight-based replication as just one strategy in their playbook. AI botnets could also act intelligently to counter defenders, but we haven’t tested this.”

GitHub has taken the step of expiring unvalidated npm trusted publishing configurations 48 hours after creation after researchers found that it was possible for an attacker to push new versions of a different user’s package without having to steal their credentials.

“Your configuration becomes validated and exempt from expiry after its first successful publish,” the Microsoft-owned subsidiary said. “Changing the repository or project identity requires a new trust relationship with a fresh 48-hour validation window — ordinary edits don’t restart the deadline.”

The change contains the risk associated with trusting a repository or project name that changes ownership, GitHub said, adding npm also rejects trusted publishing tokens from GitHub Actions issue_comment events going forward, along with the Existing pull_request_target restriction.

In an update to its documentation, GitHub states that a newly created trusted publisher configuration must complete its first successful publish within two days, which validates the configuration and binds it to the “repository’s immutable identity.”

The development comes in response to findings from Bloom Security, which discovered a loophole within npm’s trusted publishing to take over thousands of packages and trigger a Shai-Hulud-style cascading supply chain attack. The findings were reported to GitHub on June 6, 2026.

“Imagine this: an attacker compromises a single package. An AI agent on a popular package maintainer’s device autonomously pulls it for a project,” security researcher Golan Myers said. “That single endpoint is used to push malicious code to the popular package. All the package users are exposed. All because one person with the correct access was exposed.”

Trusted publishing was introduced in npm in July 2025 as a credential-free authentication method that allows developers to push npm packages from a CI/CD platform, such as GitHub Actions, using OpenID Connect (OIDC) tokens.

When trusted publishing is configured, the developer’s CI/CD provider generates a short-lived cryptographic token for a specific workflow run that the npm registry verifies against a pre-authorized configuration.

This eliminates the need to store, rotate, or manually manage long-lived npm secrets, reducing the risk of credential leaks. It also automatically generates provenance attestations for npm packages, allowing users to establish where a package was built and who published it, thereby strengthening software supply chain security.

“What this research demonstrates is something the security community has seen repeatedly across supply chain attacks: the binding between a human-readable identifier and the thing it identifies is the most important invariant in any trust system. When that binding can be silently replaced – by the ordinary act of renaming a repository, not by any attacker action – the trust guarantee collapses.”

This is not the first time unconventional supply chain attack vectors have been discovered. Earlier this August, Bloom Security also found that both Open VSX and Microsoft’s VS Code Marketplace allowed extension packs to reference non-existent extensions in the marketplaces.

The technique, referred to as Extension Resurrection, can be weaponized to hijack extensions already referenced inside these trusted bundles just by claiming a namespace left unclaimed. As many as 87 packs in Open VSX and 677 packs in the VS Code Marketplace were flagged as vulnerable.

“Attackers could claim those empty slots, publish malicious code under them, and have it installed automatically alongside the trusted pack – silently pushing updates to anyone who installed the pack previously,” Myers noted at the time. “Because extensions run with full host access, this effectively amounts to [remote code execution] delivered under the guise of legitimacy.”