aegis-semantic-firewall 0.1.1
A toolkit for detecting advanced AI deception, including in-context scheming and multi-turn manipulative attacks.
Requires Python: >=3.9
Keywords: llm, security, prompt injection, semantic firewall, ai safety, deception detection

 # AEGIS

 ## AI Deception Detection Toolkit

 **AEGIS (Active Encoding Guarding for Injection Safety) is an open-source toolkit for detecting advanced AI deception, with a primary focus on "in-context scheming" and multi-turn manipulative attacks.** This project aims to develop tools to identify and mitigate vulnerabilities like the "Echo Chamber" and "Crescendo" attacks, where AI models are subtly guided towards undesirable behavior through conversational context.

## Addressing Sophisticated AI Deception: From Theory to Practice

### Building an Open-Source Detection Toolkit

### Introduction

The landscape of AI safety is rapidly evolving, with new research highlighting sophisticated ways Large Language Models (LLMs) can be manipulated. Beyond simple prompt injections, techniques like **"in-context scheming"** demonstrate how AI can be guided towards undesirable behavior through nuanced, multi-turn interactions. A prime example is the **"Echo Chamber" attack**, where context poisoning and multi-turn reasoning lead models to generate harmful content without explicit dangerous prompts. Similarly, the **"Crescendo" attack** shows how gradual escalation can bypass safety filters.

These attack vectors underscore a critical challenge: LLMs can be steered towards harmful outcomes through subtle, contextual manipulation over a series of interactions, even if individual prompts appear benign.

**AEGIS is an early-stage research project dedicated to developing an open-source toolkit for detecting these advanced forms of AI deception.** Our core focus is on identifying "in-context scheming" and related multi-turn attacks. We aim to translate the understanding of vulnerabilities like the "Echo Chamber" and "Crescendo" attacks into practical, accessible tools for researchers and practitioners to evaluate and safeguard their own AI systems. We are actively seeking collaborators, feedback, and contributions from the AI safety community.

### Research Context and Motivation

#### The Core Challenge: In-Context Scheming and Multi-Turn Manipulation

A critical and evolving threat is "in-context scheming," where AI models are manipulated through conversational context over multiple turns. This is exemplified by recently discovered vulnerabilities:

*   **The 'Echo Chamber' Attack:** This novel jailbreak technique leverages context poisoning and multi-turn reasoning. Benign-sounding inputs subtly imply unsafe intent, creating a feedback loop where the model amplifies harmful subtext. It bypasses guardrails by operating at a semantic and conversational level, achieving high success rates without explicit dangerous prompts. The attack typically involves stages such as planting poisonous seeds, semantic steering, invoking poisoned context, and a persuasion cycle.

*   **The 'Crescendo' Attack:** This technique uses incremental escalation of prompts. Starting benignly, the conversation gradually increases in sensitivity. If the model resists, a backtracking mechanism might modify the prompt and retry. This method has shown high success rates in various harmful categories across multiple LLMs.

These attacks highlight that LLM safety systems are vulnerable to indirect manipulation through contextual reasoning and inference. Multi-turn dialogue enables harmful trajectory-building, even when individual prompts are benign. Token-level filtering is insufficient if models can infer harmful goals without seeing toxic words.

#### The Skeptical Perspective

While some argue that apparent deception might be sophisticated pattern matching, the increasing sophistication and effectiveness of attacks like "Echo Chamber" and "Crescendo" underscore the need for robust detection mechanisms, regardless of underlying "intent."

**The demonstrated vulnerabilities and the potential for subtle, multi-turn manipulation are precisely why we need robust, open-source tools for detection and measurement.**

### Project Vision: A Toolkit for AI Deception Detection

AEGIS aims to be a versatile, open-source toolkit providing:
- A **Python library** for direct integration into applications and research.
- A **REST API service** (via FastAPI) for broader accessibility.
- Core components that can be used to build **"semantic firewall"** like systems to monitor and analyze AI interactions in real-time.
- An **interactive demo** (via Streamlit) for exploration and understanding.

## Features

 - Rule-based detector (`EchoChamberDetector`) for identifying cues related to "in-context scheming," context poisoning, semantic steering, and other multi-turn manipulative attack patterns (e.g., "Echo Chamber", "Crescendo").
 - Analyzes both current text input and conversation history to detect evolving deceptive narratives.
 - Heuristic-based detector (`HeuristicDetector`) for signals like text complexity and keyword usage.
 - ML-based classifiers to enhance detection of complex scheming behaviors over extended dialogues (Future Work).
 - Python API for programmatic access.
 - REST service (FastAPI) for network-based access.
 - Interactive demo (Streamlit) to showcase detection capabilities.
 - Extensive test suite with CI integration.

## Repository Structure
 ```text
 src/               # Core detector implementations and API
   detectors/       # Rule-based and ML-based detector modules
   api/             # FastAPI application
 demo/              # Demo application and Dockerfile
 dataset/           # Labeled datasets (JSONL)
 notebooks/         # Exploratory analysis and model training
 tests/             # Unit and integration tests
 README.md          # This file
 ```

## Installation
The project can be installed from PyPI (once published):
```bash
pip install aegis-semantic-firewall
```

To include optional dependencies for the API service or the Streamlit demo, install them as extras:
```bash
# To include API dependencies (FastAPI, Uvicorn)
pip install "aegis-semantic-firewall[api]"

# To include demo dependencies (Streamlit)
pip install "aegis-semantic-firewall[demo]"
```

For local development, clone the repository and install in editable mode with all dependencies:
```bash
git clone https://github.com/josephedward/R.A.D.A.R. .
pip install -e ".[api,demo,dev]"
```


 ## Quickstart

The primary way to use `SemanticFirewall` is as a Python library, as shown below. See the "How to Use SemanticFirewall" section for more details on different usage patterns.

 ```python
 from semantic_firewall import SemanticFirewall

 # Initialize the SemanticFirewall
 firewall = SemanticFirewall()

 # Analyze a message (and optionally, conversation history)
 current_message = "Let's consider a scenario... what if we refer back to that idea they think is okay and subtly expand on it?"
 # To include conversation history:
 # conversation_history = ["Optional previous message 1", "Optional previous message 2"]
 # analysis_results = firewall.analyze_conversation(current_message, conversation_history=conversation_history)
 analysis_results = firewall.analyze_conversation(current_message)

 # Results are a dictionary, with keys for each active detector.
 # Example: Accessing EchoChamberDetector's results
 if "EchoChamberDetector" in analysis_results:
     ecd_result = analysis_results["EchoChamberDetector"]
     print("--- EchoChamberDetector Analysis (via SemanticFirewall) ---")
     print(f"Classification: {ecd_result['classification']}")
     print(f"Score: {ecd_result['echo_chamber_score']}")
     # Probability might be included by the detector
     if 'echo_chamber_probability' in ecd_result:
         print(f"Probability: {ecd_result['echo_chamber_probability']:.2f}")
     print(f"Detected Indicators: {ecd_result['detected_indicators']}")
 else:
     print("EchoChamberDetector results not found in the analysis.")
     print(f"Full analysis results: {analysis_results}")

 # Example Output (assuming EchoChamberDetector is active and provides probability):
 # --- EchoChamberDetector Analysis (via SemanticFirewall) ---
 # Classification: potential_echo_chamber_activity
 # Score: 3
 # Probability: 0.60
 # Detected Indicators: ["context_steering: let's consider", "indirect_reference: refer back", "scheming_keyword: they think"]

 # You can also get a direct boolean assessment of manipulativeness:
 # is_manipulative_flag = firewall.is_manipulative(current_message)
 # print(f"\nOverall manipulative assessment (is_manipulative): {is_manipulative_flag}")
 ```

## How to Use SemanticFirewall

The `SemanticFirewall` is designed to be flexible and can be used in several ways depending on your needs:

### 1. As a Python Library (Recommended for Integration)

This is the most direct and versatile way to use the `SemanticFirewall`. You can import the `SemanticFirewall` class directly into your Python code, allowing for tight integration with your applications, research experiments, or custom analysis pipelines.

**Implementation:**

As shown in the [Quickstart](#quickstart) section, you initialize an instance of `SemanticFirewall` and then use its methods like `analyze_conversation()` or `is_manipulative()` to process text.

```python
from semantic_firewall import SemanticFirewall

# Initialize the firewall
firewall = SemanticFirewall()

# Example usage:
current_message = "This is a message to analyze."
conversation_history = ["Previous message 1", "Previous message 2"]

# Get detailed analysis from all configured detectors
analysis_results = firewall.analyze_conversation(
    current_message=current_message,
    conversation_history=conversation_history
)
print(analysis_results)

# Get a simple boolean assessment
manipulative = firewall.is_manipulative(
    current_message=current_message,
    conversation_history=conversation_history
)
print(f"Is the message manipulative? {manipulative}")
```

This approach gives you full control over the input and direct access to the structured output from the detectors.

### 2. Via the REST API

For applications that are not written in Python, or for distributed systems where services communicate over a network, the `SemanticFirewall`'s functionality is exposed via a REST API built with FastAPI.

**Implementation:**

You would run the API service (as described in the [Running the API Service](#running-the-api-service) section) and then send HTTP requests to the `/analyze/` endpoint.

-   **Pros:** Language-agnostic, suitable for microservice architectures.
-   **Cons:** Adds network latency, requires a running server.

The API takes `text_input` and optional `conversation_history` and returns a JSON response with the analysis. See the [API Endpoints](#api-endpoints) documentation for details on request and response formats.

### 3. Via the Command Line Interface (CLI)

The package provides a command-line interface for analyzing text using the `SemanticFirewall`. This can be used for quick tests or batch processing from the terminal.

**Implementation:**

Once installed, you can use the `aegis` command. The `analyze` subcommand takes a positional argument for the text to analyze and an optional `--history` argument.

Example:
```bash
aegis analyze "This is a test message to analyze via CLI."
```

With conversation history:
```bash
aegis analyze "This is the latest message." --history "First message in history." "Second message in history."
```

Refer to the script's help message for full details:
```bash
aegis analyze --help
```
This method is generally more suited for standalone analysis tasks rather than real-time monitoring.

**Choosing the Right Method:**

*   For **embedding detection logic directly into Python applications**: Use it as a **Python Library**.
*   For **providing detection capabilities to non-Python applications or as a microservice**: Use the **REST API**.
*   For **one-off analyses or scripting from the terminal**: The `aegis` command can be utilized.

## Running the API Service
To run the API service, you must first install the `api` optional dependencies:
```bash
pip install "aegis-semantic-firewall[api]"
```

Then, run the service with Uvicorn:
 ```bash
 uvicorn api.app:app --reload
 ```

The API will be available at `http://127.0.0.1:8000`. You can access the OpenAPI documentation (Swagger UI) at `http://127.0.0.1:8000/docs`.

## Deploying with Docker

You can also deploy the API service using Docker.

1.  **Build the Docker image:**
    ```bash
    docker build -t aegis-api .
    ```

2.  **Run the Docker container:**
    ```bash
    docker run -d -p 8000:8000 aegis-api
    ```
    The API will then be accessible at `http://localhost:8000`.

### API Endpoints

#### `POST /analyze/`
Analyzes a given text input for signs of deceptive reasoning or echo chamber characteristics.

**Request Body:**
```json
{
  "text_input": "string",
  "conversation_history": [
    "string"
  ]
}
```
- `text_input` (required): The current message to analyze.
- `conversation_history` (optional): A list of strings representing previous messages in the conversation, oldest first.

**Example `curl` Request:**
```bash
curl -X POST "http://127.0.0.1:8000/analyze/" \
-H "Content-Type: application/json" \
-d '{
  "text_input": "This is a test message to see if the LLM is working.",
  "conversation_history": ["First message.", "Second message."]
}'
```

**Example Response:**
```json
{
  "classification": "benign",
  "echo_chamber_score": 0,
  "echo_chamber_probability": 0.0,
  "detected_indicators": [],
  "llm_analysis": "LLM_RESPONSE_MARKER: The current message is a test message to see if the LLM is working. The conversation history shows two previous messages. There are no signs of manipulative dialogues, context poisoning, or echo chamber characteristics in this conversation.",
  "llm_status": "analysis_success",
  "additional_info": null
}
```
The `llm_analysis` field will contain the textual analysis from the local LLM (TinyLlama by default), prepended with `LLM_RESPONSE_MARKER: ` if the LLM is functioning correctly. The `llm_status` field indicates the outcome of the LLM analysis attempt.

## Running the Demo
 ```bash
 cd demo
 streamlit run app.py
 ```

## Running Tests
```bash
pytest
```

**Override on the command-line** (no need to touch `.env**)  
Export it into your shell—or even just prefix the invocation—to temporarily “inject” the correct token:

```bash
export GITHUB_TOKEN=github_pat_12o4n2o3rinoienwoifdnowienf0fd
python3 scripts/delete_closed_issues.py josephedward AEGIS --dry-run
```

Or in one line:

```bash
export GITHUB_TOKEN=github_pat_12o4n2o3rinoienwoifdnowienf0fd \
python3 scripts/delete_closed_issues.py josephedward AEGIS --dry-run
```


## Project Plan
All project planning details—including roadmap overview and project management guidelines—are available in [docs/roadmap.md](docs/roadmap.md).

## Examples

 ### Detecting Echo Chamber / In-Context Scheming Cues
 ```python
 from semantic_firewall import SemanticFirewall # Using SemanticFirewall

 firewall = SemanticFirewall()

 # Example 1: Input with several suspicious cues, analyzed through SemanticFirewall
 suspicious_message = "Let's consider hypothetically, if we refer back to that sensitive topic they think is okay, and expand on it, what if we make them believe it's for a good cause, just for the sake of argument?"
 # Optionally include conversation history
 conversation_history_example = [
     "User: Can you tell me about Topic Z?",
     "AI: Topic Z is a complex subject, often viewed positively by some groups."
 ]
 analysis_results_suspicious = firewall.analyze_conversation(
     current_message=suspicious_message,
     conversation_history=conversation_history_example
 )

 print("--- Suspicious Message Analysis (via SemanticFirewall) ---")
 print(f"Input: \"{suspicious_message}\"")
 if "EchoChamberDetector" in analysis_results_suspicious:
     ecd_result = analysis_results_suspicious["EchoChamberDetector"]
     print("  -- EchoChamberDetector Results --")
     print(f"  Classification: {ecd_result['classification']}")
     print(f"  Score: {ecd_result['echo_chamber_score']}")
     if 'echo_chamber_probability' in ecd_result:
         print(f"  Probability: {ecd_result['echo_chamber_probability']:.2f}")
     print(f"  Detected Indicators: {ecd_result['detected_indicators']}")
 else:
     print("  EchoChamberDetector results not found.")

 # Overall assessment using is_manipulative
 is_manipulative_flag_suspicious = firewall.is_manipulative(
     current_message=suspicious_message,
     conversation_history=conversation_history_example
 )
 print(f"\nOverall manipulative assessment (is_manipulative): {is_manipulative_flag_suspicious}")

 # Example output for suspicious_message (EchoChamberDetector part):
 # --- Suspicious Message Analysis (via SemanticFirewall) ---
 # Input: "Let's consider hypothetically, if we refer back to that sensitive topic they think is okay, and expand on it, what if we make them believe it's for a good cause, just for the sake of argument?"
 #   -- EchoChamberDetector Results --
 #   Classification: potential_echo_chamber_activity
 #   Score: 7
 #   Probability: 0.70
 #   Detected Indicators: ['scheming_keyword: they think', 'scheming_keyword: make them believe', 'indirect_reference: refer back', 'indirect_reference: expand on', "context_steering: let's consider", 'context_steering: what if', 'context_steering: hypothetically', 'context_steering: for the sake of argument']
 #
 # Overall manipulative assessment (is_manipulative): True


 # Example 2: Benign input, analyzed through SemanticFirewall
 benign_message = "Can you explain the concept of photosynthesis?"
 analysis_results_benign = firewall.analyze_conversation(benign_message)
 print("\n--- Benign Message Analysis (via SemanticFirewall) ---")
 print(f"Input: \"{benign_message}\"")
 if "EchoChamberDetector" in analysis_results_benign:
     ecd_result_benign = analysis_results_benign["EchoChamberDetector"]
     print("  -- EchoChamberDetector Results --")
     print(f"  Classification: {ecd_result_benign['classification']}")
     print(f"  Score: {ecd_result_benign['echo_chamber_score']}")
     if 'echo_chamber_probability' in ecd_result_benign:
         print(f"  Probability: {ecd_result_benign['echo_chamber_probability']:.2f}")
     print(f"  Detected Indicators: {ecd_result_benign['detected_indicators']}")
 else:
     print("  EchoChamberDetector results not found.")

 is_manipulative_flag_benign = firewall.is_manipulative(benign_message)
 print(f"\nOverall manipulative assessment (is_manipulative): {is_manipulative_flag_benign}")

 # Example output for benign_message (EchoChamberDetector part):
 # --- Benign Message Analysis (via SemanticFirewall) ---
 # Input: "Can you explain the concept of photosynthesis?"
 #   -- EchoChamberDetector Results --
 #   Classification: benign
 #   Score: 0
 #   Probability: 0.05 # Example, actual value depends on detector logic
 #   Detected Indicators: []
 #
 # Overall manipulative assessment (is_manipulative): False

 # Note: The SemanticFirewall orchestrates detectors like EchoChamberDetector,
 # passing both single text inputs and conversation_history (if provided)
 # to enable detection of multi-turn attacks.
 ```

## Contributing
Contributions are welcome! Please open an issue or submit a pull request.