joanitolopo/ankh-base-convbert-q8

Ankh Base ConvBERT Q8

This model is a downstream Q8 secondary structure prediction head trained on frozen Ankh Base embeddings.

Architecture

• Frozen backbone: Ankh Base
• Downstream head: ConvBERTForMultiClassClassification
• Input dim: 768
• Hidden dim: 384
• Attention heads: 4
• Kernel size: 7
• Dropout: 0.2
• Number of labels: 8

Training Status

• Last completed epoch: 5
• Best validation Q8 accuracy: 0.39778582010894004

Important Note

This repository contains only the downstream head. For inference, load Ankh Base first, extract embeddings, then pass the embeddings to this head.