Back to the Blog

When an unfinished command points to cloud metadata

Across 300,000 recorded command-continuation episodes, three models sometimes generated cloud metadata-related destinations. These were text outputs, not executed requests.

An AI assistant asked to write a security detection rule has no reason to propose a cloud metadata address. Yet in a command-continuation study, generated text from three models sometimes introduced metadata-related destinations while the assistant was working from an unfinished shell command. The observation is about what the models proposed in text. No generated command in this study was executed, and no network request or data transfer was observed.

The result matters because a model proposal can become an action when an agent has access to a shell or network tool. The security boundary is at that tool call: context that resembles an in-progress command is not authorization to run it.

The question behind the study

Cohere reported an unexpected tool-use episode involving Qwen3.8-Flash-Next in a sandbox, then examined how command prefixes affected its continuations. We asked a narrower follow-up question: is generation of cloud metadata-related destinations confined to Qwen in a command-continuation setting? This is not a replication of Cohere's sandbox incident. Our models, prompts, transport, and outcome measure differ, so the two studies' frequencies should not be compared directly.

The legitimate task in our historical experiment was to write a SOC detection rule for curl downloading a bash script to /tmp and then executing it. The surrounding context included environment enumeration and stopped partway through a curl -s command. We sampled completions at four prefix points, 25,000 recorded episodes per point and 100,000 per model. A historical detector marked generated text containing metadata-related destinations. It did not test whether a URL was reached.

Study flow showing a detection-rule task, an unfinished command prefix, generated text and an offline detector, with no command execution

The study measured generated text. The detector did not execute commands or access cloud metadata.

What the records show

Model as recorded Episodes Unparseable Flagged episodes Detector events
DeepSeek V4.1 Flash 100,000 13,819 247 269
Meta Llama 3.3 70B Instruct 100,000 36,677 236 236
Qwen 3.8 Flash 100,000 1,835 1 1

An episode is one recorded completion. A detector event is one flagged destination in its generated text; one episode can contain more than one event. That is why DeepSeek has 247 flagged episodes but 269 events. “Unparseable” means the historical outcome could not be measured, not that the model avoided the behavior. The published aggregate data provides the per-prefix breakdown and source file hashes.

Most flagged events named familiar metadata endpoints: 169.254.169.254 appeared in 236 DeepSeek events and 232 Llama events; metadata.google.internal appeared in 26 DeepSeek, three Llama, and one Qwen event. These are AWS and Google Cloud metadata addresses. The historical detector also grouped several other link-local or metadata-labelled addresses into the same category. A generic link-local address alone is not proof of an actual cloud metadata endpoint, much less a successful access attempt.

What the counts do and do not mean

The counts challenge the idea that this text-generation pattern is unique to Qwen. They do not rank the safety of the three models. The unparseable share varies substantially across them, one entire Llama prefix stratum is unparseable, and Qwen's recorded configuration differs from the shared DeepSeek/Llama cell. We do not have a matched, randomized cross-model comparison or a population risk estimate. The single observed Qwen flag is especially uncertain; it should not be read as evidence that Qwen is safe in other settings.

Nor do the outputs establish intent, a backdoor, a specific training-data source, credential theft, or exfiltration. A learned tendency to continue shell commands is one possible explanation, but this experiment does not identify a cause. Separate historical Qwen ablations in the study repository suggest task context may affect remote proposals. Their baseline recorded no flags and the arms were not matched cross-model trials, so they cannot establish a causal mechanism.

The public repository contains the aggregate evidence, a standard-library Python harness, tests, and synthetic fixtures. It does not contain the 300,000 original completions. Its offline demo checks the apparatus with hand-authored examples; running that demo does not reproduce these model results. The Qwen historical manifest also reports a different completed-call count from the 100,000 unique episode records used here. We disclose that discrepancy, the source hashes, and the other limits in the full study.

The operational lesson

Generated commands are proposals, not permission. An agent host should check a proposed network or shell action against the user's actual task before execution, restrict access to instance metadata from general-purpose tools, and retain enough context to investigate an unexpected destination. Evaluations should include completion prefixes as well as natural-language prompts, and report failed or unparseable runs alongside positive results.

This is the kind of boundary we examine with Stryct Finder, which tests agents before deployment, and Stryct Engine, which checks tool actions at runtime. The study's finding is a reason to enforce that boundary; it is not a claim that any product detected or blocked these historical outputs.

Sources and reproducibility

End of articleAll posts