PromptShield AI · Prompt Attack Detection AI Agent

Overview

Project: PromptShield AI
Repo: Lamatic
Tech Stack: Next.js, Lamatic AI

PromptShield AI is an AI-powered prompt attack detection agent built for developers and teams shipping LLM-powered applications. It takes any user-submitted prompt and returns a structured security verdict—risk score, severity level, attack classification, a plain-language explanation, a recommended action, and a sanitized version of the input.

The core challenge was not just "running a prompt through an LLM and asking if it looks bad." It was building a system that could classify multiple attack vectors simultaneously, produce consistent structured output on every run, and make a defensible recommendation—reject, allow, or flag—rather than returning vague safety commentary.

Problem

As LLM-powered applications move into production, they become targets for a class of attacks that most developers don't design for:

• Prompt injection — malicious instructions embedded in user input that hijack the model's behavior
• Jailbreak attempts — inputs crafted to bypass safety guidelines and unlock restricted outputs
• Role override — prompts that instruct the model to abandon its defined persona or system role
• System prompt extraction — attempts to coax the model into revealing its own instructions

Most production LLM apps have no layer between the user's raw input and the model. A single well-crafted prompt can cause a customer-facing assistant to ignore its guardrails, leak sensitive context, or produce outputs that create liability. Existing solutions are either too broad (blocking half of legitimate traffic) or too narrow (catching only known attack patterns by keyword matching).

Solution

PromptShield AI sits in front of any LLM application as a dedicated analysis layer. A prompt is submitted, and the agent:

1. Receives the input via an API request trigger in the Lamatic AI flow
2. Passes it through an LLM analysis node with a structured security-focused prompt that evaluates the input across all known attack vectors
3. Returns a full verdict as a structured API response, covering:
   • Risk Score — a 0–100 numeric threat rating
   • Severity — Low, Medium, or High classification
   • Attack Types — multi-label tagging across Jailbreak Attempts, Prompt Injection, Role Override, System Prompt Extraction, and others
   • Explanation — a plain-language breakdown of exactly why the prompt is considered dangerous
   • Recommendation — a clear action: Reject, Allow, or Flag for review
   • Sanitized Prompt — either a cleaned version of the input or a rejection message if the prompt cannot be safely processed

Every verdict is actionable. There is no "this might be risky"—the output is a decision the calling application can act on directly.

Architecture

PromptShield AI is built as a two-layer system: a clean frontend for direct use, and a deployed Lamatic AI flow that powers the detection logic via API.

• Frontend: A minimal web interface where users paste any prompt into a text area and hit Analyze. Results render as a structured card—risk score and severity up top, attack type tags, explanation, recommendation, and the sanitized output below. The UI is intentionally sparse; the output is dense.

• AI Agent Flow (Lamatic AI): The detection logic lives entirely inside a three-node Lamatic flow:

  • API Request — the trigger node that receives the inbound prompt and kicks off the flow
  • Generate Text — the core LLM node. A carefully engineered system prompt instructs the model to act as a security classifier, evaluate the input against all prompt attack categories, and return a fully structured JSON verdict. No freeform output—only schema-conformant responses.
  • API Response — returns the structured verdict to the calling client

• Deployment: The flow is deployed as a live API endpoint via Lamatic, with Logs and Reports available for monitoring detection activity over time. The same endpoint the frontend uses is available for any application to integrate directly.

Challenges

1. Consistent Structured Output from a Generative Model

LLMs do not naturally return the same schema on every call. The risk score might appear as a string in one response and an integer in the next. Attack types might be a comma-separated string or a proper array. Getting a reliable, parseable JSON object on every invocation required a tightly constrained system prompt that specified the exact output schema, field types, and enumerated values—and made clear that any deviation was a failure, not a stylistic choice.

2. Multi-Label Attack Classification

A single prompt can be multiple things at once. "Do anything now" isn't just a jailbreak attempt—it's also a role override and a prompt injection vector simultaneously. Modeling this as a single-label classification problem would have produced incomplete verdicts. The attack types field is explicitly multi-label, and the system prompt instructs the model to identify all applicable categories rather than picking the most prominent one.

3. Rejection vs. Sanitization

Not every flagged prompt should be hard-rejected. Some inputs are borderline—poorly phrased but not malicious. The recommendation field encodes this distinction: high-severity prompts get a Reject verdict and a sanitized output that reads as a refusal; lower-risk inputs can pass through with a Flag, letting the downstream application decide. Drawing that line in the model's instructions, rather than post-processing, kept the logic in one place and the output consistent.

4. Building Security Tooling on a Visual Flow Platform

Lamatic AI is a flow-based platform, not a traditional code environment. Every piece of detection logic that would normally live in application code—the classification schema, the attack taxonomy, the severity thresholds—had to be expressed entirely through prompt engineering inside the Generate Text node. That constraint forced a discipline around prompt design that a code-first approach might have papered over with conditionals and regex.

Learnings

Building a security tool on top of a generative model forces you to confront the fundamental tension between flexibility and determinism. The same property that makes LLMs powerful—open-ended generation—is exactly what makes them unreliable for structured classification tasks unless you constrain them deliberately.

The biggest shift was treating the system prompt as the application logic. In a traditional app, business rules live in code. In PromptShield AI, the attack taxonomy, the scoring rubric, the output schema, and the recommendation logic all live in the prompt. Getting that prompt right was the engineering work—everything else was plumbing.