model taking too much system memory
#1
by
Vinitrajputt
- opened
hey @TheBloke , from when the autogptq has merged into the transformers the system memory cunsumption has been increased too much. like before it only takes around 6-7 gb system ram to load a 13b model at 4bit but from now its taking too much and not fitting into the clould notebooks(kaagle and colab) system ram. is there any solution for that?