Regarding the model type
Can you check the model type once as tested with starcoder not loading the model from Ctranformer , also tried with StarCoder , modely_type = gpt_bigcode still not loading the model?
Actually I guess CTransformers doesn't support Starcoder/BigCode yet. llama.cpp added StarCoder support on September 15th and Ctransformers hasn't been updated since September 10th.
Please raise an issue on CTransformers asking for Starcoder support. I will edit my README to remove mention of ctransformers for now.
Sure
But as i see the base line model architecture type : "architectures": [
"GPTBigCodeForCausalLM"
] , "model_type": "gpt_bigcode", ,is there any possibility to load the model using hugging face in 4 BIt ?
Sure, you can load the GPTQ models in 4bit - see my GPTQ repo which includes instructions on how to load it from Python
Great i could able to load the GPTQ model and also ggml model with llamcpp Thanks.