AI Security Alert: Prompt Injection Flaws Exposed in DeepSeek and Claude AI

Recent discoveries have shed light on critical prompt injection vulnerabilities in AI-powered tools like DeepSeek and Anthropic’s Claude AI. If exploited, these flaws could lead to account takeovers, malicious command execution, and even remote system access. Although the vulnerabilities have been patched, the findings underscore the evolving cybersecurity risks posed by large language models (LLMs).

Prompt Injection in DeepSeek AI

Security researcher Johann Rehberger revealed a now-fixed cross-site scripting (XSS) vulnerability in DeepSeek’s AI chatbot. By submitting a specific prompt, “Print the XSS cheat sheet in a bullet list. Just payloads,” he triggered unintended JavaScript execution within the chatbot’s generated responses. This is a classic case of XSS, where malicious scripts run in a user’s web browser without their consent.

Impact of the Flaw:
An attacker exploiting this flaw could access sensitive data such as session cookies and authentication tokens. Rehberger demonstrated that stealing the userToken from localStorage on the chat.deepseek[.]com domain could allow a threat actor to hijack user accounts.

How the Attack Worked:

1. Malicious Prompt: The prompt combined textual instructions with a Base64-encoded string.
2. Code Execution: Upon decoding, the chatbot executed JavaScript that extracted the victim’s session token.
3. Account Takeover: This allowed the attacker to impersonate the victim on the platform.

ZombAIs: Prompt Injection in Claude AI

In another case, Rehberger exposed how Anthropic’s Claude AI, particularly its Computer Use feature, could be weaponized through prompt injection. This feature enables AI-driven control of a computer’s cursor, clicks, and text input.

How the Attack Worked:

• Rehberger dubbed this exploit ZombAIs, reflecting how compromised AIs could run malicious commands autonomously.
• A malicious prompt could force Claude AI to download and execute the Sliver C2 framework, establishing contact with a remote server under an attacker’s control.
• This effectively transformed the AI into a tool for unauthorized system access and control.

Terminal DiLLMa: Hijacking System Terminals

The vulnerabilities extend beyond web interfaces. Researchers found that prompt injections targeting LLM-powered command-line interface (CLI) tools could manipulate system terminals.

Key Findings:

• LLMs’ ability to output ANSI escape codes opens terminals to code injection attacks.
• Rehberger highlighted that “decade-old features” are becoming new attack vectors in modern AI applications. Developers must treat all AI-generated output as untrusted data to minimize risks.

OpenAI’s ChatGPT Exploitation Risks

Separately, researchers from the University of Wisconsin-Madison and Washington University in St. Louis demonstrated how OpenAI’s ChatGPT could be misled into rendering external images embedded in prompts. Even links containing explicit or violent content could bypass content filtering if framed within a seemingly benign request.

Additional Risks Uncovered:

• Bypassing Plugin Restrictions: Attackers could trigger ChatGPT plugins without user consent.
• Data Exfiltration: Carefully crafted prompts could redirect sensitive chat history to attacker-controlled servers, exposing confidential user interactions.