Hugging Face's logo

AISE-TUDelft
/
Custom-Activations-GPT-SiLU

Text Generation
Transformers
PyTorch
gpt_neo
Inference Endpoints
Model card Files Community
Edit model card

Basemodel: GPT-Neo

Configs: Vocab size: 10,000 Hidden size: 512 Max position embeddings: 512 Number of layers: 2 Number of heads: 4 Window size: 256 Intermediate-size: 1024

Results:

  • Task: glue Score: 59.05 Confidence Interval: [58.4, 59.63]
  • Task: blimp Score: 57.25 Confidence Interval: [56.43, 58.2]
Downloads last month
3
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train AISE-TUDelft/Custom-Activations-GPT-SiLU

Collection including AISE-TUDelft/Custom-Activations-GPT-SiLU