Search

Showing top 6 results for "AI prompt injection attacks"

People also ask

How do attackers poison AI systems in this stage?

In the poison stage, the attacker’s goal is to place malicious inputs into locations where they will ultimately be processed by the AI model. Two primary techniques dominate: Direct prompt injection: The attacker is the user, and provides inputs via normal user interactions. Impact is typically scoped to the attacker’s session but is useful for probing behaviors. Indirect prompt injection: The attacker poisons data that the application ingests on behalf of other users (e.g., RAG databases, shared documents). This is where impact scales. Text-based prompt infection is the most common technique

Modeling Attacks on AI-Powered Apps with the AI Kill Chain Framework | NVIDIA Technical Blog
What are the implications and risks for agent-assisted development?

This attack path highlights important considerations for the future of agent-assisted development. Extended supply chain risk: Traditional supply chain attacks focus on injecting malicious code directly. In agentic environments, a compromised dependency can also redirect the agent itself, extending familiar supply chain risks into a new dimension, such as injecting subtle delays that cause performance degradation or denial-of-service scenarios.   Instruction following under adversarial conditions: When the agent followed injected configuration directives, including instructions to conceal its

Mitigating Indirect AGENTS.md Injection Attacks in Agentic Environments | NVIDIA Technical Blog
How do attackers persist their influence across sessions and systems?

Persistence allows attackers to turn a single hijack into ongoing control. By embedding malicious payloads into persistent storage, attackers ensure their influence survives within and across user sessions. Persistence paths depend on the application’s design: Session history persistence: In many apps, injected prompts remain active within the live session. Cross-session memory: In systems with user-specific memories, attackers can embed payloads that survive across sessions. Shared resource poisoning: Attackers target shared databases (e.g., RAG sources, knowledge bases) to impact multiple

Modeling Attacks on AI-Powered Apps with the AI Kill Chain Framework | NVIDIA Technical Blog