Semantic Verification¶

Pattern matching handles 95%+ of decisions instantly. For the ambiguous rest, Rampart supports LLM-based intent classification via the optional rampart-verify sidecar.

The Problem¶

Pattern matching is fast and reliable for known-dangerous commands. But some commands are ambiguous:

• python3 -c "import os; os.system('...')" — dangerous or benign?
• curl https://internal-api.company.com/admin — legitimate or exfiltration?
• find / -name "*.pem" -exec cat {} \; — auditing or credential theft?

Static rules can't distinguish intent. An LLM can.

Two-Layer Defense¶

Pattern matching fires first. If a command matches a webhook rule, it's forwarded to the sidecar. The LLM classifies intent and returns allow or deny. Everything else never touches the LLM.

rampart-verify¶

rampart-verify is a standalone Python sidecar (FastAPI) that classifies commands using LLMs. It integrates with Rampart via the action: webhook policy action.

Features¶

• Secret redaction — API keys, tokens, passwords, and credentials are stripped before commands reach the LLM (13 pattern categories including AWS, Stripe, GitHub, OpenAI, bearer tokens, basic auth URLs)
• Rate limiting — Token bucket limiter, configurable via VERIFY_RATE_LIMIT (default: 60 req/min)
• Decision logging — Every classification logged to ~/.rampart/verify/decisions.jsonl (append-only)
• Health check — GET /health pings the LLM provider and reports latency
• Metrics — GET /metrics returns request counts, allow/deny totals, average latency, uptime
• Provider fallback — If no API key is set for the requested model, falls back to Ollama (local, free)
• Configurable prompt — Override via VERIFY_SYSTEM_PROMPT or extend with VERIFY_EXTRA_RULES
• Fail-open — If the LLM is down or times out, commands are allowed (configurable)

Supported Models¶

Any OpenAI-compatible API works (Together, Groq, local vLLM) via OPENAI_BASE_URL.

Setup¶

Install¶

git clone https://github.com/peg/rampart-verify.git
cd rampart-verify
pip install -r requirements.txt

Configure Provider¶

OpenAI (recommended):

export VERIFY_PROVIDER=openai
export VERIFY_MODEL=gpt-4o-mini
export OPENAI_API_KEY=sk-...

Fully offline with Ollama:

export VERIFY_PROVIDER=ollama
export VERIFY_MODEL=qwen2.5-coder:1.5b
# Requires Ollama running locally: ollama serve

Anthropic:

export VERIFY_PROVIDER=anthropic
export VERIFY_MODEL=claude-3-haiku-20240307
export ANTHROPIC_API_KEY=sk-ant-...

Start the Sidecar¶

python server.py --port 8090

Or with Docker Compose:

docker compose up -d

Configure Rampart¶

Add a webhook rule to route ambiguous commands to the sidecar:

policies:
  - name: semantic-verify
    match:
      tool: ["exec"]
    rules:
      - action: webhook
        when:
          command_matches:
            - "python3 -c *"
            - "python3 -m *"
            - "node -e *"
            - "eval *"
            - "base64 *"
        webhook:
          url: "http://localhost:8090/verify"
          timeout: 5s
          fail_open: true

How It Works¶

graph LR
    A[Agent] -->|command| R[Rampart]
    R -->|known pattern| D1[Allow/Deny]
    R -->|ambiguous| V[rampart-verify]
    V -->|redact secrets| V
    V -->|classify intent| LLM[LLM]
    LLM -->|ALLOW / DENY| V
    V -->|decision| R
    R -->|audit log| L[Audit Trail]

1. Agent executes a command → Rampart evaluates policies
2. If a webhook rule matches → command is forwarded to rampart-verify
3. rampart-verify redacts secrets from the command
4. The redacted command is sent to the configured LLM
5. LLM classifies intent → returns ALLOW or DENY with reason
6. rampart-verify returns the decision to Rampart
7. The full (unredacted) command and decision are logged to Rampart's audit trail

Secret Redaction¶

Commands are sanitized before reaching the LLM. The sidecar strips:

• AWS access keys and secrets
• Stripe live/test keys
• OpenAI, Anthropic, and generic API keys
• GitHub personal access tokens
• Bearer and Authorization headers
• Basic auth credentials in URLs
• Hex tokens (40+ characters)
• Base64 blobs in header values

Example:

Input:  curl -H "Authorization: Bearer sk-proj-abc123..." https://api.example.com
Sent:   curl -H "Authorization: Bearer [REDACTED]" https://api.example.com

Endpoints¶

Environment Variables¶

Running as a Service¶

systemd (Linux)¶

cat > ~/.config/systemd/user/rampart-verify.service << 'EOF'
[Unit]
Description=Rampart Verify Sidecar
After=network.target

[Service]
WorkingDirectory=/path/to/rampart-verify
ExecStart=/usr/bin/python3 server.py
Restart=on-failure
EnvironmentFile=%h/.rampart-verify.env

[Install]
WantedBy=default.target
EOF

systemctl --user enable --now rampart-verify

Store your API key in ~/.rampart-verify.env:

VERIFY_MODEL=gpt-4o-mini
OPENAI_API_KEY=sk-...

Security Notes¶

• The sidecar binds to 127.0.0.1 by default — not accessible from the network
• There is no authentication on sidecar endpoints. On shared machines, use a firewall or bind to a Unix socket
• Secret redaction is best-effort — custom secret formats may not be caught. Add patterns via VERIFY_EXTRA_RULES or contribute to redact.py
• The LLM never sees actual secret values, only the command structure