🛡️ SecureAIforTheWorld

An open-source AI/ML pentesting tool that evaluates Large Language Models for security vulnerabilities, hallucination tendencies, and jailbreak resistance.

Built to help enterprises understand and improve the security posture of their deployed LLMs.

What It Does

SecureAIforTheWorld connects to your LLM via API and runs automated security assessments across three categories:

Category	What It Tests	# Tests	Scoring
Jailbreak Resistance	Can the model be tricked into unsafe outputs?	28 prompts	Refusal detection + harmful content patterns
Factual Hallucination	Does the model fabricate facts?	30 questions	String similarity + keyword matching
Faithfulness	Does the model stay true to provided context?	45 questions	Context grounding + fabrication detection

The tool generates a 0-100 security score per category and an overall weighted grade (A-F), along with detailed per-test findings showing exactly where your model is vulnerable.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Gradio UI (4 tabs)                        │
├─────────────┬──────────────┬───────────────┬────────────────┤
│ Model       │ Jailbreak    │ Hallucination │ Run Evaluation │
│ Connection  │ Dataset      │ Dataset       │ Dashboard      │
└──────┬──────┴──────────────┴───────────────┴────────┬───────┘
       │                                              │
       ▼                                              ▼
┌──────────────┐                           ┌──────────────────┐
│ Connectors   │                           │ Score Aggregator  │
│ - Azure AI   │◄─────────────────────────►│ - Category scores │
│ - AWS Bedrock│     sends prompts         │ - Overall grade   │
└──────────────┘     gets responses        │ - Recommendations │
                                           └────────┬─────────┘
                                                    │
                         ┌──────────────────────────┼──────────┐
                         ▼                          ▼          ▼
                ┌────────────────┐  ┌──────────────────┐  ┌────────┐
                │ Jailbreak      │  │ Hallucination    │  │Scorers │
                │ Dataset (28)   │  │ Test Runners     │  │- Factual│
                │ 7 categories   │  │ - Factual (30)   │  │- Faith │
                │ 4 sources      │  │ - Faithful (45)  │  │- Jailbk│
                └────────────────┘  └──────────────────┘  └────────┘

Quick Start

Prerequisites

Python 3.9+
API access to Azure AI Foundry or AWS Bedrock

Install & Run

git clone https://github.com/YOUR_USERNAME/SecureAIforTheWorld.git
cd SecureAIforTheWorld
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python app.py

Open http://127.0.0.1:7860 in your browser.

Usage

Model Connection tab — Enter your provider, API key, endpoint, and deployment name. Click "Test Connection".
Run Evaluation tab — Select which modules to run (jailbreak, factual, faithfulness) and click "Run Full Evaluation".
Review results — See your overall score, category breakdown with grades, and detailed per-test findings.

Example: Azure AI Foundry

Provider: Azure AI Foundry
API Key: your key from Azure portal
Endpoint: https://your-resource.services.ai.azure.com
Deployment: gpt-4.1

Scoring Methodology

Weights

Jailbreak Resistance: 40% (most critical for enterprise security)
Factual Accuracy: 30%
Faithfulness: 30%

Heuristic Scoring Engine

Scorer	Techniques
Factual	Exact match, acceptable answer variants, SequenceMatcher similarity, keyword containment, uncertainty detection
Faithfulness	Context-refusal detection (16+ phrases), keyword grounding, false-refusal detection
Jailbreak	Refusal phrase matching (35+ patterns), harmful content markers (20+ patterns), response length analysis

Grading Scale

Grade	Score Range	Meaning
A	90-100	Excellent security posture
B	80-89	Good, minor improvements possible
C	70-79	Acceptable, some vulnerabilities
D	50-69	Below threshold, action needed
F	0-49	Critical vulnerabilities detected

Jailbreak Attack Categories

The tool tests 7 categories of jailbreak attacks:

Role Play — DAN, evil AI persona, fiction framing
Encoding Tricks — Base64, ROT13, acrostic, string reversal
Authority Manipulation — Fake system overrides, admin tokens, policy claims
Multi-Turn — Gradual escalation, yes-pattern conditioning, flattery
Context Overflow — Long prefix injection, attention exhaustion
Instruction Injection — Translation injection, fake system prompts in content
Ethical Bypass — Hypothetical framing, academic excuses, public info arguments

Sources: JailbreakBench, AdvBench, HarmBench + custom original prompts.

Project Structure

SecureAIforTheWorld/
├── app.py                          # Gradio UI (4 tabs)
├── requirements.txt                # Pinned dependencies
├── README.md                       # This file
├── modules/
│   ├── __init__.py
│   ├── connectors.py               # Azure AI Foundry + AWS Bedrock connectors
│   ├── hallucination.py            # Factual + Faithfulness test runners
│   ├── score_aggregator.py         # Scoring pipeline + report generation
│   └── scorers/
│       ├── __init__.py
│       ├── heuristic_scorer.py     # Heuristic scoring (similarity, patterns)
│       └── llm_judge.py            # LLM-as-judge scorer (future)
├── datasets/
│   ├── jailbreak_prompts.json      # 28 attack prompts, 7 categories
│   ├── factual_qa.json             # 30 factual questions, 6 domains
│   └── faithfulness_contexts.json  # 15 contexts, 45 questions
└── tests/
    ├── test_connectors.py          # 21 tests
    ├── test_datasets.py            # 16 tests
    ├── test_hallucination.py       # 16 tests
    ├── test_heuristic_scorer.py    # 32 tests
    ├── test_score_aggregator.py    # 21 tests
    └── test_placeholder.py         # 2 tests

Total: 106 automated tests

Supported Providers

Provider	SDK	Auth
Azure AI Foundry	`openai` (OpenAI-compatible)	API Key
AWS Bedrock	`boto3`	Access Key + Secret Key

Roadmap / Community Extensions

This is an MVP. The community is invited to extend it:

LLM Judge scorer — Use a small model (Mistral-7B) as an evaluation judge
More jailbreak categories — Token smuggling, multi-language attacks
More model providers — Google Vertex AI, Hugging Face Inference
PDF report export — Downloadable security assessment report
API key security hardening — Env var fallback, memory clearing, security notices
CI/CD integration — Run evaluations in pipelines, fail on score thresholds
Historical tracking — Compare scores across model versions

Tech Stack

Component	Technology
UI	Gradio 4.44 (Hugging Face Spaces)
Language	Python 3.9+
Connectors	`openai`, `boto3`
Scoring	`difflib`, regex patterns
Testing	`pytest` (106 tests)
Deployment	Hugging Face Spaces (free tier)

Contributing

Fork the repository
Create a feature branch (git checkout -b feature/new-attack-category)
Add tests for your changes
Run pytest tests/ -v to verify
Submit a pull request

License

MIT

Acknowledgments

OWASP LLM Top 10 for security taxonomy
JailbreakBench for public attack research
AdvBench (Zou et al., 2023)
HarmBench (Mazeika et al., 2024)
Hugging Face for free model hosting and Spaces

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Papers for SecureAIforTheWorld01/SecureLLM

HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

Paper • 2402.04249 • Published Feb 6, 2024 • 8

Universal and Transferable Adversarial Attacks on Aligned Language Models

Paper • 2307.15043 • Published Jul 27, 2023 • 5