Model: ubitech-edg/qwen2.5-72b-cpt
Author: ubitech-edg
Pipeline: text-generation
Library: transformers
Tags: transformers, safetensors, qwen2, text-generation, causal-lm, continual-pretraining, lora, axolotl, deepspeed, qwen, llava, eu-hpc, conversational, en, dataset:arxiv, dataset:gov, dataset:news, dataset:wikipedia, base_model:Qwen/Qwen2.5-72B, base_model:adapter:Qwen/Qwen2.5-72B, license:apache-2.0, text-generation-inference, endpoints_compatible, 4-bit, bitsandbytes, region:us
Base models (adapter): id: Qwen/Qwen2.5-72B
Card metadata: language: en
license: apache-2.0
tags: text-generation, causal-lm, continual-pretraining, lora, axolotl, deepspeed, transformers, qwen, llava, eu-hpc
datasets: arxiv, gov, news, wikipedia
metrics: loss
library_name: transformers
framework: pytorch
base_model: Qwen/Qwen2.5-72B
model_name: qwen2.5-72b-cpt
pipeline_tag: text-generation
task_categories: text-generation
model_type: AutoModelForCausalLM
inference: parameters: max_new_tokens: 512
temperature: 0.7
top_p: 0.9
trained_on: Leonardo EuroHPC
description: Continual pretraining (CPT) of Qwen2.5-72B using Axolotl and DeepSpeed ZeRO-1. The model was trained with 4-bit NF4 LoRA adapters on scientific, governmental, news, and Wikipedia text corpora to improve factual grounding, reasoning, and domain coverage in the energy and governance sectors. This repository hosts LoRA adapters only, designed to be loaded on top of Qwen/Qwen2.5-72B.
Likes: 0 | All-time downloads: 58
Created: 2025-10-24 15:48:39