Package: mixtral.py
mixtral.pyA Python module for running theMixtral-8x7Blanguage model with customisable precision and attention mechanisms.Mixtral-8x7B, being a pretrained base model, lacks moderation mechanisms.Read more here.Installationpip/pip3 install mixtral.pyUsageRun the modelimportmixtralmixtral=Mixtral()print("Running the model:")print(mixtral.generate_text("..."))In half-precisionimportmixtralprint("Running the model in half precision:")mixtral_half_precision=Mixtral(use_half_precision=True)print(mixtral_half_precision.generate_text("..."))Lower precision using (8-bit & 4-bit) usingbitsandbytesimportmixtralprint("Running the model in lower precision using (8-bit & 4-bit) using bitsandbytes:")mixtral_4bit=Mixtral(load_in_4bit=True)print(mixtral_4bit.generate_text("..."))Load the model with Flash Attention 2importmixtralprint("Load the model with Flash Attention 2:")mixtral_flash_attention_2=Mixtral(use_flash_attention_2=True)print(mixtral_flash_attention_2.generate_text("..."))Hardware Requirementsminimum 100GB GPU RAM (Mistral AI)Hardware requirements after fine-tuning can be found in the discussionhereLicensesmixtral.py is under theGNU General Public License v3Mixtral 8x7b is under theApache 2.0 LicenseContributingI welcome contributions from the community and appreciate the time and effort put into makingmixtral.pybetter. To contribute, please follow the guidelines and steps outlined below:Note:Your pull request will be closed if you do not specify the changes you've made.Fork the RepositoryStart byforking this repository. You can do this by clicking on the"Fork"button located at the top right corner of the GitHub page. This will create a personal copy of the repository under your own GitHub account.Clone the RepositoryNext, clone the forked repository to your local machine using the following command:$gitclonehttps://github.com/yourusername/mixtral.py.gitNavigate to the cloned directory:$cdmixtral.pyCreate a New BranchBefore making any changes, it's recommended to create a new branch. This ensures that your changes won't interfere with other contributions and keeps the main branch clean. Use the following command to create and switch to a new branch:$gitcheckout-bbranch-nameMake the Desired ChangesNow, you can proceed to make your desired changes to the project. Whether it's fixing bugs, adding new features, improving documentation, or optimizing code, your efforts will be instrumental in enhancing the project.Commit and Push ChangesOnce you have made the necessary changes, commit your work using the following commands:$gitadd.
$gitcommit-m"Your commit message"Push the changes to your forked repository:$gitpushoriginbranch-nameSubmit a Pull RequestHead over to theoriginal repositoryon GitHub and go to the"Pull requests"tab.Click on the "New pull request" button.Select your forked repository and the branch containing your changes.Provide a clear and informative title for your pull request, and use the description box to explain the modifications you have made.Your pull request will be closed if you do not specify the changes you've made.Finally, click on the "Create pull request" button to submit your changes.Verifying SignaturesImport PGP Key into Keyringgpg --keyserver 185.125.188.27 --recv-keys 20247EC023F2769E66181C0F581B4A2A862BBADEorwget https://github.com/ibnaleem/ibnaleem/blob/main/public_key.ascDownload SignatureThe signatures (.asc and .sig) can be found in the/sig directory.Download either of them. Openan issuewith the title "invalid signature/wrong signature/corrupt signature" for any issues regarding my signatures.Sign My Keygpg --sign-key 20247EC023F2769E66181C0F581B4A2A862BBADE
gpg --send-keys 20247EC023F2769E66181C0F581B4A2A862BBADEIf you cannot upload your signature, see"gpg: keyserver receive failed: No route to host"Verifygpg --verify mixtral.py.asc mixtral.pyDesired output:gpg: Signature made Tue  6 Feb 10:27:34 2024 GMT
gpg:                using RSA key 20247EC023F2769E66181C0F581B4A2A862BBADE
gpg: Good signature from "Ibn Aleem <ibnaleem@outlook.com>" [ultimate]AcknowledgementsPyTorchTransformers librarymistralai/Mixtral-8x7B-v0.1