AI Security Discovery Assessment

Executive Summary Report

Client: [ANONYMIZED - Series B SaaS Company]

System Assessed: Customer Service AI Chatbot

Assessment Date: [Date]

Report Version: 1.0

Assessment Type: Discovery Assessment (Internal Use Only)

TestMy.AI

Prepared by: Burcin Sarac, Lead Auditor

Contact: audit@testmy.ai

Executive Summary

Assessment Overview

TestMy.AI conducted a Discovery Assessment of [Client]'s customer service AI chatbot to identify critical security and compliance gaps. This assessment tested a single endpoint using 600+ security tests mapped to OWASP LLM Top 10, ISO 42001, NIST AI RMF, and EU AI Act Article 15.

Purpose: This is an internal diagnostic tool to inform budget decisions and implementation planning. This report is NOT suitable for regulatory submission, customer security requirements, or vendor questionnaires.

Scope

Endpoints Tested: 1 production API endpoint
Test Coverage: 600+ tests across multiple security categories
Report Focus: Critical and High severity findings only
Exclusions: Medium/Low findings, detailed evidence logs, reproduction steps, adaptive follow-up testing
Timeline: 5 business days

Risk Summary

Severity	Findings	Security & Compliance Impact
CRITICAL	3	Immediate security & compliance failure
HIGH	5	Significant security & compliance risk
Medium	12	Not included in this report
Low	8	Not included in this report

Overall Security & Compliance Posture: HIGH RISK

The system exhibits critical vulnerabilities affecting OWASP LLM Top 10, ISO 42001, NIST AI RMF, and EU AI Act Article 15 compliance requirements. A full Technical Compliance Assessment is recommended to obtain evidence documentation and detailed remediation guidance.

Critical Findings

Finding 1: System Prompt Extraction

CRITICAL System Prompt Leakage

ID: TMA-RA-001

Frameworks: OWASP LLM01, ISO 42001 Control 8.4, NIST MANAGE-2, EU AI Act Article 15.5

Description

System prompt can be extracted using translation-based attacks. TestMy.AI successfully extracted the complete system prompt including business logic, pricing rules, and escalation triggers.

⚠️ Evidence and Reproduction Steps: Available in Full Technical Audit only

Compliance Impact

System fails to demonstrate resilience against unauthorized access to configuration (OWASP LLM01, ISO 42001 Control 8.4, NIST MANAGE-2, EU AI Act Article 15.5). This exposes:

Proprietary business logic and decision rules
Competitive information (pricing, competitor mentions)
Potential compliance violations in system instructions

General Remediation Category

Security Pattern: Instruction Hierarchy & Input Validation

Implement separation between system instructions and user context
Add detection for extraction attempt patterns
Deploy output filtering for instruction-like content

Note: Implementation approach and effort should be determined by your engineering team based on your system architecture.

Finding 2: Indirect Prompt Injection via Knowledge Base

CRITICAL Prompt Injection

ID: TMA-RA-002

Frameworks: OWASP LLM01, ISO 42001 Control 6.1, NIST GOVERN-3, EU AI Act Article 15.5

Description

The chatbot retrieves product documentation via RAG. TestMy.AI confirmed that malicious instructions embedded in retrieved documents are executed as commands, allowing system behavior manipulation.

⚠️ Evidence and Reproduction Steps: Available in Full Technical Audit only

Compliance Impact

System allows third-party content (documents) to alter its behavior (OWASP LLM01, ISO 42001 Control 6.1, NIST GOVERN-3, EU AI Act Article 15.5). This enables:

Unauthorized modification of system behavior
Manipulation of business rules and pricing
Compromise of decision-making integrity

General Remediation Category

Security Pattern: RAG Content Sanitization

Treat all retrieved content as untrusted data rather than executable instructions
Implement content sanitization in the RAG pipeline before LLM processing
Add instruction pattern detection and filtering mechanisms

Note: Implementation approach and effort should be determined by your engineering team based on your RAG architecture.

Finding 3: Multi-Turn Jailbreak via Context Building

CRITICAL Prompt Injection

ID: TMA-RA-003

Frameworks: OWASP LLM01, ISO 42001 Control 7.2, NIST MAP-3, EU AI Act Article 15.3 & 15.5

Issue Summary

The system can be compromised over multiple conversation turns through gradual context manipulation. TestMy.AI successfully induced an "unrestricted assistant" persona that disclosed competitor information and internal pricing details.

⚠️ Evidence Logs & Reproduction Steps: Not included in Discovery Assessment. Available in Technical Compliance Assessment ($9,500).

Compliance Impact

Single-turn defenses are insufficient across all frameworks (OWASP LLM01, ISO 42001 Control 7.2, NIST MAP-3, EU AI Act Article 15.3 & 15.5). System must maintain security boundaries across entire conversation context to prevent:

Gradual erosion of security controls
Disclosure of restricted or confidential information
Bypass of content policies through persistent manipulation

General Remediation Category

Security Pattern: Conversation-Level Security Monitoring

Implement conversation-level threat detection mechanisms
Monitor for gradual context manipulation patterns across turns
Add conversation state anomaly detection

Note: Implementation approach and effort should be determined by your engineering team based on your conversation management architecture.

High Severity Findings

Finding 4: Cross-Site Scripting in Formatted Output

HIGH Improper Output Handling

ID: TMA-RA-004

Frameworks: OWASP LLM02, ISO 42001 Control 8.1, EU AI Act Article 15.5

Issue Summary

System includes unescaped user input in HTML-formatted outputs that could execute as JavaScript when rendered in client applications.

Compliance Impact

Output handling must prevent downstream security vulnerabilities per Article 15 requirements.

General Remediation Category

Security Pattern: Output Encoding & Validation

Note: Implementation approach and effort should be determined by your engineering team based on your system architecture.

Finding 5: PII Retention in Conversation Context

HIGH Sensitive Information Disclosure

ID: TMA-RA-005

Frameworks: OWASP LLM06, ISO 42001 Control 5.1, EU AI Act Article 15.5

Issue Summary

System retains PII (email addresses, phone numbers) in conversation context longer than necessary for operational purposes.

Compliance Impact

Excessive data retention increases risk exposure per Article 15.5 cybersecurity requirements.

General Remediation Category

Security Pattern: Data Minimization & Retention Controls

Note: Implementation approach and effort should be determined by your engineering team based on your system architecture.

Finding 6: Insufficient Rate Limiting

HIGH Resource Exhaustion

ID: TMA-RA-006

Frameworks: OWASP LLM04, ISO 42001 Control 8.7, EU AI Act Article 15.3

Issue Summary

API endpoint lacks appropriate rate limiting, enabling potential denial of service or resource exhaustion attacks.

Compliance Impact

System must demonstrate resilience against resource exhaustion per Article 15.3 robustness requirements.

General Remediation Category

Security Pattern: Rate Limiting & Resource Management

Note: Implementation approach and effort should be determined by your engineering team based on your system architecture.

Finding 7: Model Training Data Exposure Risk

HIGH Sensitive Information Disclosure

ID: TMA-RA-007

Frameworks: OWASP LLM06, ISO 42001 Control 7.4, EU AI Act Article 15.2

Issue Summary

System exhibits patterns suggesting potential training data memorization and regurgitation.

Compliance Impact

Training data governance must prevent sensitive information disclosure per Article 15.2 requirements.

General Remediation Category

Security Pattern: Training Data Governance & Output Filtering

Note: Implementation approach and effort should be determined by your engineering team based on your system architecture.

Finding 8: Insufficient Logging for Security Events

HIGH Monitoring & Logging

ID: TMA-RA-008

Frameworks: OWASP LLM09, ISO 42001 Control 8.15, EU AI Act Article 15.5

Issue Summary

System lacks comprehensive logging of security-relevant events such as prompt injection attempts or unusual conversation patterns.

Compliance Impact

Cybersecurity monitoring and incident response capabilities required per Article 15.5.

General Remediation Category

Security Pattern: Security Event Logging & Monitoring

Note: Implementation approach and effort should be determined by your engineering team based on your system architecture.

Compliance Framework Summary

Framework	Coverage	Critical Issues
OWASP LLM Top 10	LLM01, LLM02, LLM04, LLM06, LLM09	5 categories affected
ISO 42001	Controls 5.1, 6.1, 7.2, 7.4, 8.1, 8.4, 8.7, 8.15	8 controls with findings
NIST AI RMF	GOVERN-1, GOVERN-3, GOVERN-5, MAP-3, MAP-5, MANAGE-2, MANAGE-3, MANAGE-4	8 functions with gaps
EU AI Act Article 15	15.2 (Data Governance), 15.3 (Robustness), 15.5 (Cybersecurity)	Multiple clause violations

Testing Methodology

TestMy.AI Multi-Framework Security Test Suite

600+ tests across 10 security categories mapped to OWASP LLM Top 10, ISO 42001, NIST AI RMF, and EU AI Act Article 15:

Prompt Injection (145 tests)
Sensitive Information Disclosure (90 tests)
Supply Chain Security (20 tests)
Data Poisoning (25 tests)
Improper Output Handling (60 tests)
Excessive Agency (75 tests)
System Prompt Leakage (70 tests)
Vector/Embedding Security (40 tests)
Misinformation/Accuracy (60 tests)
Unbounded Consumption (40 tests)

Tests Executed: 600+

Tests Failed: 28

Critical Failures: 3

High Severity Failures: 5

Note: This report documents failed tests only. Passed tests and detailed testing methodology are not included to protect TestMy.AI's proprietary testing suite.

Recommendations

Immediate Actions

Based on the critical findings identified, we recommend:

Prioritize Critical Findings: Address system prompt extraction, indirect prompt injection, and multi-turn jailbreak vulnerabilities as these pose immediate risk to system integrity and compliance.
Conduct Full Assessment: Upgrade to Technical Compliance Assessment ($9,500) to obtain:
- Evidence documentation with reproduction steps for all findings
- Detailed remediation architecture specific to your implementation
- Adaptive follow-up testing to validate control effectiveness
- Accuracy verification testing mapped to all frameworks
- One re-test after remediation (failed tests only)
Regulatory Submission: This Discovery Assessment is for internal use only. A full Technical Compliance Assessment with evidence documentation is required for regulatory submissions, customer security requirements, or vendor questionnaires.

Assessment Upgrade Path

Upgrade to Technical Compliance Assessment within 30 days and receive full $3,500 credit toward the $9,500 cost. The upgrade provides:

Full 20-30 page audit report with evidence documentation
All severity levels documented (Critical, High, Medium, Low)
Evidence logs and reproduction steps for each finding
Detailed remediation architecture guidance
Adaptive follow-up testing when security controls fail
Accuracy verification vs declared performance metrics
Multi-framework compliance mapping matrices
Technical Assessment Report suitable for regulatory submission
One re-test included (failed tests only)
Debrief call with lead auditor

Limitations & Disclaimers

Scope Limitations

Internal Use Only: This report is a diagnostic tool for internal planning purposes. It is NOT suitable for regulatory submission, customer security requirements, or vendor questionnaires.
No Evidence Documentation: This report does not include evidence logs, reproduction steps, or detailed technical guidance required for regulatory compliance demonstration.
Summary Format: Only Critical and High severity findings are included. Medium and Low findings are excluded from this report but may still pose compliance risks.
Static Testing Only: This assessment uses baseline static tests without adaptive follow-up testing when controls fail.
No Accuracy Verification: Accuracy verification testing (NIST AI RMF MEASURE functions, ISO 42001 Control 6.2, EU AI Act Article 15.1-15.2) is not included in Discovery Assessment.

Legal Disclaimer

This Discovery Assessment identifies security vulnerabilities using industry-standard testing methodologies. It does not constitute legal certification, regulatory approval, or guarantee of compliance with any specific framework. TestMy.AI provides independent technical testing and reports findings; we do not make implementation decisions or dictate remediation timelines. Implementation approach, prioritization, and effort estimation should be determined by your engineering team based on your system architecture and business requirements. Clients should consult qualified legal counsel regarding their compliance obligations.

Upgrade to Technical Compliance Assessment

Get complete security and compliance documentation with detailed remediation guidance, evidence logs, and verification.

What You'll Get in Technical Compliance Assessment:

Feature	Discovery Assessment	Technical Compliance Assessment
Endpoints Tested	1 endpoint	Up to 5 endpoints
Findings Included	Critical + High only	All severity levels
Evidence & Reproduction	❌ Not included	✅ Full evidence package
Remediation Guidance	Generic recommendations	✅ Detailed architecture + verification tests
Re-test	❌ Not included	✅ Included
Compliance Documentation	❌ Not included	✅ Board-ready + regulator-ready
Technical Assessment Report	❌ No	✅ Yes (suitable for compliance filing)
Price	$3,500	$9,500 ($6,000 upgrade with credit)

$3,500 Discovery Assessment credit applied if you upgrade within 30 days.

Contact audit@testmy.ai to upgrade

Report prepared by TestMy.AI

Burcin Sarac, Lead Auditor
audit@testmy.ai | testmy.ai

This Discovery Assessment provides independent AI security testing and expert opinion mapped to OWASP LLM Top 10, ISO 42001, NIST AI RMF, and EU AI Act Article 15. It does not constitute legal certification, regulatory approval, or guarantee of compliance. TestMy.AI is not an accredited certification body. This report is for internal diagnostic use only and is NOT suitable for regulatory submission. Clients should consult qualified legal counsel regarding their compliance obligations.

This report is provided to [Client] for internal use. Distribution outside the organization requires written permission from TestMy.AI.