Spaces:
Sleeping
Sleeping
File size: 5,104 Bytes
b802cbd ef12530 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 |
---
title: PerplexityViewer
emoji: 📈
colorFrom: gray
colorTo: blue
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
pinned: false
license: gpl-3.0
short_description: Simple inspection of perplexity using color-gradients
---
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
# PerplexityViewer 📈
A Gradio-based web application for visualizing text perplexity using color-coded gradients. Perfect for understanding how confident language models are about different parts of your text.
## Features
- **Dual Model Support**: Works with both decoder models (GPT, DialoGPT) and encoder models (BERT, RoBERTa)
- **Interactive Visualization**: Color-coded per-token perplexity using spaCy's displaCy
- **Configurable Analysis**: Adjustable iterations and MLM probability settings
- **Real-time Processing**: Instant analysis with cached models for faster subsequent runs
- **Multiple Model Types**:
- **Decoder Models**: Calculate true perplexity for causal language models
- **Encoder Models**: Calculate pseudo-perplexity using masked language modeling
## How It Works
- **Red tokens**: High perplexity (model is uncertain about this token)
- **Green tokens**: Low perplexity (model is confident about this token)
- **Gradient colors**: Show varying degrees of model confidence
## Installation
1. Clone this repository or download the files
2. Install dependencies:
```bash
pip install -r requirements.txt
```
## Quick Start
### Option 1: Using the startup script (recommended)
```bash
python run.py
```
### Option 2: Direct launch
```bash
python app.py
```
### Option 3: With dependency installation and testing
```bash
python run.py --install --test
```
## Usage
1. **Enter your text** in the input box
2. **Select a model** from the dropdown or enter a custom HuggingFace model name
3. **Choose model type**:
- **Decoder**: For GPT-like models (true perplexity)
- **Encoder**: For BERT-like models (pseudo-perplexity via MLM)
4. **Adjust settings** (optional):
5. **Click "Analyze"** to see the results
## Supported Models
### Decoder Models (Causal LM)
- `gpt2`, `distilgpt2`
- `microsoft/DialoGPT-small`, `microsoft/DialoGPT-medium`
- `openai-gpt`
- Any HuggingFace causal language model
### Encoder Models (Masked LM)
- `bert-base-uncased`, `bert-base-cased`
- `distilbert-base-uncased`
- `roberta-base`
- `albert-base-v2`
- Any HuggingFace masked language model
## Understanding the Results
### Perplexity Interpretation
- **Lower perplexity**: Model is more confident (text is more predictable)
- **Higher perplexity**: Model is less confident (text is more surprising)
### Color Coding
- **Green**: Low perplexity (≤ 2.0) - very predictable
- **Yellow/Orange**: Medium perplexity (2.0-10.0) - somewhat predictable
- **Red**: High perplexity (≥ 10.0) - surprising or difficult to predict
## Technical Details
### Decoder Models (True Perplexity)
- Uses next-token prediction to calculate perplexity
- Formula: `PPL = exp(average_cross_entropy_loss)`
- Each token's perplexity is based on how well the model predicted it given the previous context
### Encoder Models (Pseudo-Perplexity)
- Uses masked language modeling (MLM)
- Masks each token individually and measures prediction confidence
- Pseudo-perplexity approximates true perplexity for bidirectional models
- All content tokens are analyzed for comprehensive results
## Testing
Run the test suite to verify everything works:
```bash
python test_app.py
```
Or use the startup script with testing:
```bash
python run.py --test
```
## Configuration
The app uses sensible defaults but can be customized via `config.py`:
- Default model lists
- Processing settings
- Visualization colors and settings
- UI configuration
## Requirements
- Python 3.7+
- PyTorch
- Transformers
- Gradio 4.0+
- spaCy
- pandas
- numpy
## GPU Support
The app automatically uses GPU acceleration when available, falling back to CPU processing otherwise.
## Troubleshooting
### Common Issues
1. **Model loading errors**: Ensure you have internet connection for first-time model downloads
2. **Memory issues**: Try smaller models like `distilgpt2` or `distilbert-base-uncased`
3. **CUDA out of memory**: Reduce text length or use CPU-only mode
4. **Encoder models slow**: This is normal - each token is analyzed individually for accuracy
5. **Single analysis**: The app now performs one comprehensive analysis per run (no iterations needed)
### Getting Help
If you encounter issues:
1. Check the console output for error messages
2. Try running the test suite: `python test_app.py`
3. Ensure all dependencies are installed: `pip install -r requirements.txt`
## Examples
Try these example texts to see the app in action:
- **"The quick brown fox jumps over the lazy dog."** (Common phrase - should show low perplexity)
- **"Quantum entanglement defies classical intuition."** (Technical content - may show higher perplexity)
- **"Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo."** (Grammatically complex - interesting perplexity patterns)
|