google
/

gemma-scope-9b-pt-res

Model card Files Files and versions Community

gemma-scope-9b-pt-res / layer_0 /width_131k /average_l0_21 /hparams.json

Tom Lieberum

9B: Add sparsity lambdas for residual stream and clean up the feature splitting suite so that we only have SAEs with learning rate 7e-5

745af8a 4 days ago

27 Bytes

{"sparsity_lambda": 0.0005}