aravdhoot/ethics-llama3-8b-probes

Ethics Probes: llama3-8b

This repository contains Sparse Autoencoder (SAE) + Probe models trained on various ethics datasets.

Model

• Base Model: meta-llama/Llama-3.1-8B
• Short Name: llama3-8b

Datasets

The following datasets have been processed:

• commonsense: Best Layer: 14, Val Acc: 0.7805, Test Acc: 0.7863

Repository Structure

.
├── commonsense/
│   ├── layer_0/
│   │   ├── sae.pt
│   │   ├── probe.pt
│   │   ├── config.json
│   │   └── metrics.json
│   ├── layer_1/
│   ├── ...
│   ├── best/          # Best performing layer
│   └── README.md      # Dataset-specific details
└── README.md          # This file

Usage

Each dataset folder contains:

• layer_N/: Models for each layer
• best/: Copy of the best performing layer's models
• sae.pt: Sparse Autoencoder state dict
• probe.pt: Probe state dict
• config.json: Configuration used for training
• metrics.json: Performance metrics
• README.md: Detailed results for the dataset