not-a-feature/DNA-Embedding-Inversion

DNA Embedding Inversion

Three final LCQD decoders from How Private Are DNA Embeddings?, reconstructing DNA from mean-pooled foundation-model embeddings.

Code, embedding extraction and results

Model	Input dimensions	Extraction layer
dnabert2	768	Final hidden layer (12)
evo2	4096	blocks.26.mlp.l3 (zero-based)
ntv2	1024	Final hidden layer (29)

Each decoder covers 14 lengths from 10 to 100 nucleotides. These are the main-study checkpoints, trained on coordinate- and exact-sequence-disjoint hg38 data for 20 epochs, retaining the best validation checkpoint.

Quick start

Python 3.12; CPU or CUDA. Download this repository with Git LFS:

git lfs install
git clone https://huggingface.co/not-a-feature/DNA_Embedding_Invesion
cd DNA_Embedding_Invesion
pip install -r requirements.txt
python infer.py --model ntv2 --embeddings embeddings.npy --output reconstructions.json

embeddings.npy must contain a finite float array shaped [N, input_dimensions] of raw, unnormalized mean-pooled embeddings, extracted using the matching model and recipe linked above. The script applies the bundled exact training normalization and returns a JSON list of DNA strings. Add --device cuda for GPU inference.

Python API:

import numpy as np
from infer import Decoder

sequences = Decoder("ntv2").predict(np.load("embeddings.npy", allow_pickle=False))

No original sequence length is required. Decoding stops at the first predicted EOS; predictions are not cropped to a requested length. Tokenizers are bundled, so inference works offline after installation and download. Foundation-model weights and embedding extraction are separate.

Validation and scope

CPU predictions matched the original evaluation code on 252 embeddings: three per length, per model, for hg38 and 1000 Genomes. CLI execution, EOS handling and invalid-input checks also passed. This verifies inference parity, not a new accuracy estimate; see validation.json.

Performance depends on sequence length and embedding source. Evaluation assumes reference orientation. These checkpoints are intended for research on genomic embedding privacy.

Weights and decoder code: EUPL-1.2. Bundled tokenizers originate from DNABERT-2 and NTv2; their upstream terms apply. Configurations, normalization provenance and SHA-256 checksums are included.