File size: 992 Bytes
dc77352 c9440c3 dc77352 c9440c3 dc77352 c9440c3 dc77352 c9440c3 dc77352 c9440c3 dc77352 c9440c3 dc77352 c9440c3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
---
library_name: transformers
tags:
- arcee-ai
---
# Model Card for Model ID
This model is a dummy model created for testing purposes only. It utilizes a custom configuration to explore various training scenarios and should not be used for production.
## Configuration Highlights
The configuration of this dummy model is distinct from the original model in several key aspects:
- **Number of Layers:** Reduced to 2, allowing for quicker tests of layer-specific behaviors.
- **Experts:** Configured with 4 local experts and 2 experts per token, experimenting with the model's capacity to handle multiple expert inputs.
- **Hidden Size:** Set at 512, this smaller size is suitable for testing the impact of network width.
- **Intermediate Size:** Enlarged to 3579, to investigate how an increase in the size affects the model's ability to process information deeply. |