Module 3.2 — Polymorphic and Runtime-Generated Malware

50-minute lecture · Day 3 morning

Learning objectives

By end of this module, students can:

  1. Distinguish runtime-LLM-querying malware from conventionally-polymorphic malware, and articulate why the runtime variant is structurally harder to detect via signatures
  2. Walk the PromptLock case study correctly — including the important nuance that PromptLock was an NYU Tandon academic PoC discovered on VirusTotal, not in-the-wild ransomware
  3. Recognize HYAS Labs BlackMamba (2023) as the canonical PoC for the runtime-LLM-generated-payload pattern, and identify successors documented through 2025-2026
  4. Write network-detection signatures that catch outbound LLM-API queries from non-developer processes — the dominant detection path for this class of threat

The categorical shift

Conventionally polymorphic malware mutates its binary representation between samples while preserving its behavioral signature. Detection adapts by writing signatures on the behavior (API call sequences, system-effect patterns) rather than on the byte content.

Runtime-LLM-querying malware does something structurally different: the binary contains an API call to an LLM, and the LLM generates the malware’s next behavior at runtime. The binary itself can be small and innocuous-looking — the behavior is not in the binary at all.

This breaks several assumptions of conventional malware detection:

  1. Static signatures don’t apply. The binary doesn’t contain the malicious code; it contains the recipe for fetching the malicious code.
  2. YARA rules on the dropped payload won’t help. The “payload” is generated freshly on each execution and never written to disk in the same form twice.
  3. Behavioral analysis still works — but the relevant behaviors are the outbound LLM API call and the structured-JSON request body, not the post-execution effects (those are downstream).

The defender’s response shifts from endpoint-static-analysis to network + endpoint behavioral analysis of LLM API calls themselves.

The PromptLock case (correctly explained)

Many summaries of PromptLock describe it as “ESET’s discovery of in-the-wild AI ransomware.” This is incorrect and propagates a factual error that detection engineers should not perpetuate.

The actual story:

In late August 2025, ESET researchers discovered samples uploaded to VirusTotal that they initially flagged as “the first AI-powered ransomware in the wild.” Within days, researchers at NYU Tandon School of Engineering confirmed that they had created the samples themselves as a research proof-of-concept (PoC) called “Ransomware 3.0” — and the academic paper documenting the work was forthcoming.

The accurate description:

Sources:

Why the correction matters: The course teaches detection engineers to validate citations and not propagate received wisdom. The PromptLock case is technically real and architecturally important, but the attribution and deployment status are commonly misstated. Get them right.

The HYAS Labs BlackMamba lineage

The earlier canonical PoC for runtime-LLM-generated malware is HYAS Labs BlackMamba (2023). BlackMamba is a research PoC of a runtime-LLM-generated keylogger:

BlackMamba was specifically designed to demonstrate that static-signature detection cannot catch this pattern because the malicious code never exists in any file on disk that could be signatured.

Documented successors through 2025-2026:

The pattern is becoming common. By mid-2026, treat any binary that makes outbound calls to LLM API endpoints as suspicious until proven otherwise.

Detection signatures for runtime LLM-querying malware

The dominant detection path is network + endpoint behavioral analysis of LLM API calls. Specific signatures detection engineers should deploy:

DNS resolution from non-developer processes

alert dns (
  msg:"Possible LLM-runtime malware: API endpoint resolution from non-developer process";
  dns.query.name; content:"api.openai.com"; nocase;
  threshold: type both, track by_src, count 5, seconds 60;
  classtype:trojan-activity;
)

Tune the endpoint list to include:

TLS SNI on encrypted traffic

When SSL/TLS inspection is not available (the common case), detect at the SNI handshake level:

tls_sni; content:"api-inference.huggingface.co"; nocase;

This catches the outbound connection even when the body is encrypted.

Process-context filtering

The signal is meaningful only when paired with process context: outbound LLM API calls from python.exe, node.exe, code.exe, IDE processes, browser processes are typically legitimate (developers using copilots). Outbound LLM API calls from winword.exe, excel.exe, notepad.exe, svchost.exe, or unknown executables are highly suspicious.

A Sysmon-based detection at the endpoint:

<Rule groupRelation="and">
  <NetworkConnection>
    <DestinationHostname condition="contains">api.openai.com</DestinationHostname>
    <DestinationHostname condition="contains">api.anthropic.com</DestinationHostname>
    <DestinationHostname condition="contains">generativelanguage.googleapis.com</DestinationHostname>
  </NetworkConnection>
  <Image condition="exclude">python.exe</Image>
  <Image condition="exclude">node.exe</Image>
  <Image condition="exclude">code.exe</Image>
  <!-- ... other allow-listed developer tools ... -->
</Rule>

Structured-JSON HTTP body detection

When SSL inspection is available, detect malicious prompt content in outbound requests:

http_client_body; content:"write a python script"; nocase;
http_client_body; content:"keylogger"; nocase;
http_client_body; content:"encrypt files"; nocase;
http_client_body; content:"reverse shell"; nocase;

This is signature-based, so adversaries will encode/obfuscate. But it catches naive samples — and many in-the-wild samples are naive.

Behavioral session signatures

Long-lived outbound HTTPS sessions to LLM endpoints from server workloads (rather than user workstations) are highly suspicious. A SOC analyst’s web browser will make burst-mode calls; agent-style malware makes sustained polling calls. Tag and alert on this pattern.

Why traditional EDR struggles

Several conventional EDR detection assumptions break against runtime-LLM-querying malware:

  1. No file on disk to hash. The “payload” is in memory only. File-hash blocklists don’t apply.
  2. Anti-emulation tactics are easier. The malware doesn’t have to evade analysis — it doesn’t contain the malicious behavior to be analyzed.
  3. Behavioral analysis is delayed. The malicious effect doesn’t happen until after the LLM API returns a response, which happens seconds after the binary executes. By then the EDR’s behavioral correlation window may have moved on.
  4. Memory scanning is the only static-analysis path. And memory contents change rapidly as new prompts/responses cycle through.

The detection has to move to the network: catching the outbound API call before the malicious code is generated and executed.

Operational example: detecting LameHug / PROMPTSTEAL

LameHug (the APT28 PROMPTSTEAL malware family) is the first publicly-documented in-the-wild operational use of the runtime-LLM-query pattern. Detection engineers should build their stack against it as the canonical example.

Network signature:

Outbound HTTPS to api-inference.huggingface.co OR huggingface.co
  AND process is not in {python, node, code, IDE-tooling allow-list}
  AND user is not in {known developer group}

Endpoint signature:

Sysmon Event ID 3 (Network Connection):
  DestinationHostname matches Hugging Face inference endpoints
  Image is from %APPDATA%, %TEMP%, %USERPROFILE%\Downloads
  CommandLine contains structured JSON with prompt-like fields

Behavioral correlation:

Within 60 seconds of the Hugging Face call:
  Suspicious child process spawn (e.g., cmd.exe spawned by the binary that made the HF call)
  File system enumeration patterns
  Outbound C2 connection to non-LLM endpoint

Any one of these alone is a low-fidelity signal. The conjunction has near-zero false positives.

Discussion questions (~10 min)

  1. PromptLock was an academic PoC misreported as in-the-wild malware. Why is this distinction important for the detection engineer’s daily work? What changes about how you’d respond to a “PromptLock-class” alert vs an “academic-research-sample” alert?
  2. Your org’s developers use GitHub Copilot extensively. Outbound calls to api.openai.com from code.exe are constant and legitimate. How do you write the SIEM rule that catches PROMPTSTEAL-class malware without burying analysts in Copilot alert noise?
  3. The detection moves to the network layer. What happens when adversaries route their LLM API calls through a non-LLM CDN or a proxy that strips the destination indicator? What residual signal can you still detect?

Common mistakes

MistakeBetter approach
Citing PromptLock as in-the-wild ESET malwareCorrect attribution: NYU Tandon academic PoC; samples found on VirusTotal late August 2025
Looking for malicious code in the binaryThe code is fetched at runtime; look for the fetching in network traffic
Blocking outbound to all LLM APIsBreaks legitimate developer workflows; use process-context filtering instead
Only deploying signature-based detectionBehavioral correlation across network + endpoint context is the durable signal
Trusting any “first AI-powered X in the wild” headlineVerify discloser, verify deployment status, verify whether sample is research PoC or operational

What’s next

Module 3.3 introduces the OWASP LLM Top 10 (2025) as the defender’s checklist — a structured taxonomy of LLM-application risks, each with a concrete detection-engineering deliverable.