M4-ai

/

TinyMistral-6x248M

like 9

Yes! This model was indeed created using mlabonne’s LazyMergeKit.

That mergekit code is not working I tried a lot but it did not worked

Do you mind sharing an error code? Which code are you referring to?

fatal: destination path 'mergekit' already exists and is not an empty directory.
Installing build dependencies done

Checking if build backend supports build_editable
Getting requirements to build editable .. done
done

Installing backend dependencies . done
Preparing editable metadata (pyproject.toml)
done

Building editable for mergekit (pyproject.toml)... done
mergekit-moe config.yaml merge-copy-tokenizer-cuda--low-cpu-memory

WARNING:root:--cuda is a no-op for mergekit-moe, use "--device cuda" instead
Traceback (most recent call last):

File "/usr/local/bin/mergekit-moe", line 8, in
sys.exit(main())

File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in_call_
return self.main(*args, **kwargs)

File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)

File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)

File r/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
return callback(*args, **kwargs)

File "/content/mergekit/mergekit/options.py", line 76, in wrapper
f(*args, **kwargs)

File "/content/mergekit/mergekit/scripts/mixtral_moe.py", line 452, in main
config = MistralMOEConfig.model_validate(yaml.safe_load(config_source))

File
usr/local/lib/python3.10/dist-packages/pydantic/main.py", line 503, in model_validate
return cls. pydantic_validator.validate_python(

pydantic_core._pydantic_core.ValidationError: 1 validation error for MistralMOEConfig
experts

Field required [type=missing, input_value={'slices': [{'sources': [...}]}, 'dtype': 'float16'}, input_type=dict] information visit https://errors.pydantic.dev/2.5/v/missing

Did you choose the Mixtral branch? If you did, it may be something you need to take to Maxime Labonne because the config may not have been pushed to the readme correctly.