In the poison stage, the attacker’s goal is to place malicious inputs into locations where they will ultimately be processed by the AI model. Two primary techniques dominate: Direct prompt injection: The attacker is the user, and provides inputs via normal user interactions. Impact is typically scoped to the attacker’s session but is useful for probing behaviors. Indirect prompt injection: The attacker poisons data that the application ingests on behalf of other users (e.g., RAG databases, shared documents). This is where impact scales. Text-based prompt infection is the most common technique
Modeling Attacks on AI-Powered Apps with the AI Kill Chain Framework | NVIDIA Technical Blog… Indirect prompt injection as a supply chain vector: The agent’s summarization model was also susceptible to indirect prompt injection through code comments, illustrating how these techniques can chain together across agentic workflows. …
… The AI Kill Chain provides a clear, actionable way to break down how these attacks unfold—stage by stage. It helps teams move beyond generalized “prompt injection” concerns to see where, how, and why attackers can escalate their control. …
… Based on the NVIDIA AI Red Team ’s experience, the following mandatory controls mitigate the most serious attacks that can be achieved with indirect prompt injection: Network egress controls: Blocking network access to arbitrary sites prevents exfiltration of data or establishing a remote shell wit… …
… The VLM prompt is static: “should I stop or go?” but the attacker has some level of control over the input image. We are also only focused on open-box attacks where the attacker has access to the complete model and input prompt during development to generate their adversarial input. …
… Safety and Security Use NeMo Agent Toolkit safety and security middleware features to Red Team agentic workflows and find points of exploitability and vulnerabilities like prompt injection, jail break, tool poisoning, and other custom attacks. …
… Every prompt injection is a potential credential leak. …