Based on jondurbin/mpt-30b-qlora-compatible, however with a Bugfix for multi GPU support.