GALAXY-16B-v1.0

Technical notes

• 72 layers,DUS procedure, mistral(32)->SOLAR(48)->GALAXY(72)
• 16B parameters
• model created as an extension of depth upscaling procedure used for SOLAR by upstage

Results

• model can and will produce NSFW content
• waiting for eval results

Prompt template

• Alpaca
• chat template is embedded in tokenizer config, should load automatically

Context size

• 4096

All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel: