|
--- |
|
base_model: facebook/llm-compiler-7b-ftd |
|
license: other |
|
tags: |
|
- llama-cpp |
|
- gguf-my-repo |
|
extra_gated_prompt: "**Meta Large Language Model Compiler (LLM Compiler) LICENSE AGREEMENT**\n\ |
|
Version Release Date: 27th June 2024\n“**Agreement**” means the terms and conditions\ |
|
\ for use, reproduction, distribution and modification of the LLM Compiler Materials\ |
|
\ set forth herein.\n“**Documentation**” means the specifications, manuals and documentation\ |
|
\ accompanying the LLM Compiler distributed by Meta at:\n* [https://huggingface.co/facebook/llm-compiler-7b](https://huggingface.co/facebook/llm-compiler-7b)\ |
|
\ * [https://huggingface.co/facebook/llm-compiler-7b-ftd](https://huggingface.co/facebook/llm-compiler-7b-ftd)\ |
|
\ * [https://huggingface.co/facebook/llm-compiler-13b](https://huggingface.co/facebook/llm-compiler-13b)\ |
|
\ * [https://huggingface.co/facebook/llm-compiler-13b-ftd](https://huggingface.co/facebook/llm-compiler-13b-ftd)\n\ |
|
“**Licensee**” or “**you**” means you, or your employer or any other person or entity\ |
|
\ (if you are entering into this Agreement on such person or entity’s behalf), of\ |
|
\ the age required under applicable laws, rules or regulations to provide legal\ |
|
\ consent and that has legal authority to bind your employer or such other person\ |
|
\ or entity if you are entering in this Agreement on their behalf.\n“**Meta Large\ |
|
\ Language Model Compiler” and “LLM Compiler**” mean the foundational large language\ |
|
\ models and software and algorithms, including machine-learning model code, trained\ |
|
\ model weights, inference-enabling code, training-enabling code, fine-tuning enabling\ |
|
\ code and other elements of the foregoing distributed by Meta at:\n* [https://huggingface.co/facebook/llm-compiler-7b](https://huggingface.co/facebook/llm-compiler-7b)\ |
|
\ * [https://huggingface.co/facebook/llm-compiler-7b-ftd](https://huggingface.co/facebook/llm-compiler-7b-ftd)\ |
|
\ * [https://huggingface.co/facebook/llm-compiler-13b](https://huggingface.co/facebook/llm-compiler-13b)\ |
|
\ * [https://huggingface.co/facebook/llm-compiler-13b-ftd](https://huggingface.co/facebook/llm-compiler-13b-ftd)\n\ |
|
“**LLM Compiler Materials**” means, collectively, Meta’s proprietary LLM Compiler\ |
|
\ and Documentation (and any portion thereof) made available under this Agreement.\n\ |
|
“**Meta**” or “**we**” means Meta Platforms Ireland Limited (if you are located\ |
|
\ in or, if you are an entity, your principal place of business is in the EEA or\ |
|
\ Switzerland) and Meta Platforms, Inc. (if you are located outside of the EEA or\ |
|
\ Switzerland). \nBy clicking “I Accept” below or by using or distributing any portion\ |
|
\ or element of the LLM Compiler Materials, you agree to be bound by this Agreement.\n\ |
|
1. **License Rights and Redistribution**. \\\n\n a. <span style=\"text-decoration:underline;\"\ |
|
>Grant of Rights</span>. You are granted a non-exclusive, worldwide, non-transferable\ |
|
\ and royalty-free limited license under Meta’s intellectual property or other rights\ |
|
\ owned by Meta embodied in the LLM Compiler Materials to use, reproduce, distribute,\ |
|
\ copy, create derivative works of, and make modifications to the LLM Compiler Materials.\ |
|
\ \n\n b. <span style=\"text-decoration:underline;\">Redistribution and Use</span>.\ |
|
\ \n\n i. If you distribute or make available the LLM Compiler Materials (or\ |
|
\ any derivative works thereof), or a product or service that uses any of them,\ |
|
\ including another AI model, you shall (A) provide a copy of this Agreement with\ |
|
\ any such LLM Compiler Materials; and (B) prominently display “Built with LLM Compiler”\ |
|
\ on a related website, user interface, blogpost, about page, or product documentation.\ |
|
\ If you use the LLM Compiler Materials to create, train, fine tune, or otherwise\ |
|
\ improve an AI model, which is distributed or made available, you shall also include\ |
|
\ “LLM Compiler” at the beginning of any such AI model name.\n\n ii. If you receive\ |
|
\ LLM Compiler Materials, or any derivative works thereof, from a Licensee as part\ |
|
\ of an integrated end user product, then Section 2 of this Agreement will not apply\ |
|
\ to you. \n\n iii. You must retain in all copies of the LLM Compiler Materials\ |
|
\ that you distribute the following attribution notice within a “Notice” text file\ |
|
\ distributed as a part of such copies: “LLM Compiler is licensed under the LLM\ |
|
\ Compiler License, Copyright © Meta Platforms, Inc. All Rights Reserved.”\n\n \ |
|
\ iv. Your use of the LLM Compiler Materials must comply with applicable laws\ |
|
\ and regulations (including trade compliance laws and regulations) and adhere to\ |
|
\ the Acceptable Use Policy for Llama Materials (available at https://llama.meta.com/llama3/use-policy),\ |
|
\ which is hereby incorporated by reference into this Agreement.\n\n v. You will\ |
|
\ not use the LLM Compiler Materials or any output or results of the LLM Compiler\ |
|
\ Materials to improve any other large language model. \n\n2. **Additional Commercial\ |
|
\ Terms**. If, on the LLM Compiler release date, the monthly active users of the\ |
|
\ products or services made available by or for Licensee, or Licensee’s affiliates,\ |
|
\ is greater than 700 million monthly active users in the preceding calendar month,\ |
|
\ you must request a license from Meta, which Meta may grant to you in its sole\ |
|
\ discretion, and you are not authorized to exercise any of the rights under this\ |
|
\ Agreement unless or until Meta otherwise expressly grants you such rights. \n\ |
|
3**. Disclaimer of Warranty**. UNLESS REQUIRED BY APPLICABLE LAW, THE LLM COMPILER\ |
|
\ MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS,\ |
|
\ WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS ALL WARRANTIES OF ANY KIND,\ |
|
\ BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE,\ |
|
\ NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE\ |
|
\ SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING\ |
|
\ THE LLM COMPILER MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE\ |
|
\ LLM COMPILER MATERIALS AND ANY OUTPUT AND RESULTS.\n4. **Limitation of Liability**.\ |
|
\ IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY,\ |
|
\ WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING\ |
|
\ OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL,\ |
|
\ INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE\ |
|
\ BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.\n5. **Intellectual Property**.\n\ |
|
\n a. No trademark licenses are granted under this Agreement, and in connection\ |
|
\ with the LLM Compiler Materials, neither Meta nor Licensee may use any name or\ |
|
\ mark owned by or associated with the other or any of its affiliates, except as\ |
|
\ required for reasonable and customary use in describing and redistributing the\ |
|
\ LLM Compiler Materials or as set forth in this Section 5(a). Meta hereby grants\ |
|
\ you a license to use LLM Compiler (the “Mark”) solely as required to comply with\ |
|
\ the last sentence of Section 1.b.i. You will comply with Meta’s brand guidelines\ |
|
\ (currently accessible at[ https://about.meta.com/brand/resources/meta/company-brand/)](https://about.meta.com/brand/resources/meta/company-brand/).\ |
|
\ All goodwill arising out of your use of the Mark will inure to the benefit of\ |
|
\ Meta. \n\n b. Subject to Meta’s ownership of LLM Compiler Materials and derivatives\ |
|
\ made by or for Meta, with respect to any derivative works and modifications of\ |
|
\ the LLM Compiler Materials that are made by you, as between you and Meta, you\ |
|
\ are and will be the owner of such derivative works and modifications.\n\n c.\ |
|
\ If you institute litigation or other proceedings against Meta or any entity (including\ |
|
\ a cross-claim or counterclaim in a lawsuit) alleging that the LLM Compiler Materials\ |
|
\ or LLM Compiler outputs or results, or any portion of any of the foregoing, constitutes\ |
|
\ infringement of intellectual property or other rights owned or licensable by you,\ |
|
\ then any licenses granted to you under this Agreement shall terminate as of the\ |
|
\ date such litigation or claim is filed or instituted. You will indemnify and hold\ |
|
\ harmless Meta from and against any claim by any third party arising out of or\ |
|
\ related to your use or distribution of the LLM Compiler Materials.\n\n6. **Term\ |
|
\ and Termination**. The term of this Agreement will commence upon your acceptance\ |
|
\ of this Agreement or access to the LLM Compiler Materials and will continue in\ |
|
\ full force and effect until terminated in accordance with the terms and conditions\ |
|
\ herein. Meta may terminate this Agreement if you are in breach of any term or\ |
|
\ condition of this Agreement. Upon termination of this Agreement, you shall delete\ |
|
\ and cease use of the LLM Compiler Materials. Sections 3, 4 and 7 shall survive\ |
|
\ the termination of this Agreement. \n7. **Governing Law and Jurisdiction**. This\ |
|
\ Agreement will be governed and construed under the laws of the State of California\ |
|
\ without regard to choice of law principles, and the UN Convention on Contracts\ |
|
\ for the International Sale of Goods does not apply to this Agreement. The courts\ |
|
\ of California shall have exclusive jurisdiction of any dispute arising out of\ |
|
\ this Agreement. " |
|
extra_gated_fields: |
|
First Name: text |
|
Last Name: text |
|
Date of birth: date_picker |
|
Country: country |
|
Affiliation: text |
|
I accept the terms and conditions: checkbox |
|
geo: ip_location |
|
? By clicking Submit below I accept the terms of the license and acknowledge that |
|
the information I provide will be collected stored processed and shared in accordance |
|
with the Meta Privacy Policy |
|
: checkbox |
|
extra_gated_description: The information you provide will be collected, stored, processed |
|
and shared in accordance with the [Meta Privacy Policy](https://www.facebook.com/privacy/policy/). |
|
extra_gated_button_content: I Accept Meta LLM Compiler License and AUP |
|
--- |
|
|
|
# aifeifei798/llm-compiler-7b-ftd-Q5_K_M-GGUF |
|
This model was converted to GGUF format from [`facebook/llm-compiler-7b-ftd`](https://huggingface.co/facebook/llm-compiler-7b-ftd) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space. |
|
Refer to the [original model card](https://huggingface.co/facebook/llm-compiler-7b-ftd) for more details on the model. |
|
|
|
## Use with llama.cpp |
|
Install llama.cpp through brew (works on Mac and Linux) |
|
|
|
```bash |
|
brew install llama.cpp |
|
|
|
``` |
|
Invoke the llama.cpp server or the CLI. |
|
|
|
### CLI: |
|
```bash |
|
llama-cli --hf-repo aifeifei798/llm-compiler-7b-ftd-Q5_K_M-GGUF --hf-file llm-compiler-7b-ftd-q5_k_m.gguf -p "The meaning to life and the universe is" |
|
``` |
|
|
|
### Server: |
|
```bash |
|
llama-server --hf-repo aifeifei798/llm-compiler-7b-ftd-Q5_K_M-GGUF --hf-file llm-compiler-7b-ftd-q5_k_m.gguf -c 2048 |
|
``` |
|
|
|
Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well. |
|
|
|
Step 1: Clone llama.cpp from GitHub. |
|
``` |
|
git clone https://github.com/ggerganov/llama.cpp |
|
``` |
|
|
|
Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux). |
|
``` |
|
cd llama.cpp && LLAMA_CURL=1 make |
|
``` |
|
|
|
Step 3: Run inference through the main binary. |
|
``` |
|
./llama-cli --hf-repo aifeifei798/llm-compiler-7b-ftd-Q5_K_M-GGUF --hf-file llm-compiler-7b-ftd-q5_k_m.gguf -p "The meaning to life and the universe is" |
|
``` |
|
or |
|
``` |
|
./llama-server --hf-repo aifeifei798/llm-compiler-7b-ftd-Q5_K_M-GGUF --hf-file llm-compiler-7b-ftd-q5_k_m.gguf -c 2048 |
|
``` |
|
|