Update for Transformers GPTQ support

Browse files

Files changed (4) hide show

README.md +70 -63
config.json +32 -22
baichuan-vicuna-7b-GPTQ-4bit-128g.no-act.order.safetensors → model.safetensors +2 -2
quantize_config.json +7 -6

README.md CHANGED Viewed

@@ -4,17 +4,20 @@ license: other
 ---
 <!-- header start -->
-<div style="width: 100%;">
-    <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
 </div>
 <div style="display: flex; justify-content: space-between; width: 100%;">
     <div style="display: flex; flex-direction: column; align-items: flex-start;">
-        <p><a href="https://discord.gg/Jq4vkcDakD">Chat & support: my new Discord server</a></p>
     </div>
     <div style="display: flex; flex-direction: column; align-items: flex-end;">
-        <p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
     </div>
 </div>
 <!-- header end -->
 # Fire Balloon's Baichuan Vicuna 7B GPTQ
@@ -125,11 +128,12 @@ It was created with group_size 128 to increase inference accuracy, but without -
   * Parameters: Groupsize = 128. Act Order / desc_act = False.
 <!-- footer start -->
 ## Discord
 For further support, and discussions on these models and AI in general, join us at:
-[TheBloke AI's Discord server](https://discord.gg/Jq4vkcDakD)
 ## Thanks, and how to contribute.
@@ -144,12 +148,15 @@ Donaters will get priority support on any and all AI/LLM/model questions and req
 * Patreon: https://patreon.com/TheBlokeAI
 * Ko-Fi: https://ko-fi.com/TheBlokeAI
-**Special thanks to**: Luke from CarbonQuill, Aemon Algiz, Dmitriy Samsonov.
-**Patreon special mentions**: Mano Prime, Fen Risland, Derek Yates, Preetika Verma, webtim, Sean Connelly, Alps Aficionado, Karl Bernard, Junyu Yang, Nathan LeClaire, Chris McCloskey, Lone Striker, Asp the Wyvern, Eugene Pentland, Imad Khwaja, trip7s trip, WelcomeToTheClub, John Detwiler, Artur Olbinski, Khalefa Al-Ahmad, Trenton Dambrowitz, Talal Aujan, Kevin Schuppel, Luke Pendergrass, Pyrater, Joseph William Delisle, terasurfer , vamX, Gabriel Puliatti, David Flickinger, Jonathan Leane, Iucharbius , Luke, Deep Realms, Cory Kujawski, ya boyyy, Illia Dulskyi, senxiiz, Johann-Peter Hartmann, John Villwock, K, Ghost , Spiking Neurons AB, Nikolai Manek, Rainer Wilmers, Pierre Kircher, biorpg, Space Cruiser, Ai Maven, subjectnull, Willem Michiel, Ajan Kanaga, Kalila, chris gileta, Oscar Rangel.
 Thank you to all my generous patrons and donaters!
 <!-- footer end -->
 # Original model card: Fire Balloon's Baichuan Vicuna 7B
@@ -205,66 +212,66 @@ Please refer to [discussions 2](https://huggingface.co/fireballoon/baichuan-vicu
 ```
 Q: How can I improve my time management skills?
-A: There are many different ways to improve your time management skills. Here are a few strategies that you may find helpful:
-1 Prioritize your tasks: Make a list of all the tasks you need to complete and prioritize them based on importance and urgency. This will help you focus on the most important tasks and avoid getting bogged down by less critical items.
-2 Use a time management planner: A planner can help you organize your time by breaking down your tasks into smaller, more manageable chunks. This can help you stay focused and on track.
-3 Set clear boundaries: Setting clear boundaries between work time and personal time can help you avoid distractions and stay focused on your tasks. This may involve designating certain times of day or days of the week for work and other times for rest and leisure.
-4 Take care of your physical and mental health: Taking care of your physical and mental health can help you be more productive and focused. This may involve getting enough sleep, eating a healthy diet, and taking breaks to relax and recharge.
 Remember that improving your time management skills takes practice and patience. Don't be discouraged if you make mistakes or experience setbacks. Keep trying and you will eventually develop better time management habits that will help you be more productive and successful.
 ```
 ```
 Q: Can you help me write a formal email to a potential business partner proposing a joint venture?
-A: Sure! Here is a sample formal email that you could use as a template to propose a joint venture with a
-potential business partner:
-Subject: Proposal for Joint Venture
-Dear [Business Partner],
-I hope this email finds you well. I am writing to propose a joint venture between our companies. We believe that a collaboration between our companies could bring significant benefits to both of us, and we would like to explore the possibility of forming a partnership.
-Our companies have complementary strengths, and we believe that a joint venture could allow us to leverage those strengths to create new products, services, and opportunities. We believe that a joint venture could also provide increased revenue and growth for both of our companies.
-In order to explore the possibility of a joint venture, we would like to schedule a call to discuss the details of our proposal. During the call, we would like to discuss the terms of the joint venture, including the specific roles and responsibilities of each partner, the financial terms of the agreement, and any other relevant details.
-I would be happy to schedule a call with you at your convenience. In the meantime, please feel free to contact me with any questions or concerns you may have about our proposal.
-Sincerely,
 [Your Name]
 ```
 ```
 Q: Implement a binary search algorithm to find a specific element in a sorted array.
-A: Here is an implementation of the binary search algorithm to find a specific element in a sorted array:
- public static int binarySearch(int[] arr, int target) {
-     int left = 0;
-     int right = arr.length - 1;
-     while (left <= right) {
-         int mid = left + (right - left) / 2;
-         if (arr[mid] < target) {
-             left = mid + 1;
-         } else if (arr[mid] > target) {
-             right = mid - 1;
-         } else {
-             return mid;
-         }
-     }
-     return -1; // target not found
- }
-Here is how the binary search algorithm works:
- 1 Initialize two pointers, left and right, to the beginning and end of the array, respectively.
- 2 While the left pointer is less than or equal to the right pointer, perform the following steps:
-    1 Calculate the middle index, mid, by dividing the left pointer and the right pointer.
-    2 Check if the middle element, arr[mid], is less than or equal to the target value.
-    3 If the middle element is less than the target value, update the left pointer to mid + 1.
-    4 If the middle element is greater than the target value, update the right pointer to mid - 1.
-    5 If the middle element is equal to the target value, the target element has been found, so return the middle index.
- 3 If the target element is not found, return -1.
-This algorithm has a runtime complexity of O(log n) and a space complexity of O(1), making it an efficient way to find a specific element in a sorted array.
 ```
 ---
@@ -290,23 +297,23 @@ A: 目前，中国通用的是119火警电话。
 ```
 Q: 晚上睡不着应该怎么办
-A:失眠是一种症状，而不是一种疾病。治疗失眠的目标是解决其潜在原因。失眠的一些常见原因包括：
-*睡眠不足
-*压力
-*心理健康问题 (焦虑，抑郁等)
-*药物 (处方和非处方)
-*环境因素 (噪音，光线，温度等)
-以下是一些通常可以帮助您睡眠的方法：
-*睡觉前4-6个小时避免刺激物 (咖啡因，尼古丁，酒精)
-*睡前冥想以减少压力和焦虑
-*睡觉前不要看亮屏幕 (电脑，手机，平板电脑等)
-*睡觉前至少一个小时不要运动
-*睡觉前不要吃食物
-*在睡觉前放松并平静下来 (看看书，喝杯牛奶等)
-如果这些技术不起作用，或者如果失眠是由药物或其他医学状况引起的，您应该联系您的医生。
 ```

 ---
 <!-- header start -->
+<!-- 200823 -->
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
 </div>
 <div style="display: flex; justify-content: space-between; width: 100%;">
     <div style="display: flex; flex-direction: column; align-items: flex-start;">
+        <p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://discord.gg/theblokeai">Chat & support: TheBloke's Discord server</a></p>
     </div>
     <div style="display: flex; flex-direction: column; align-items: flex-end;">
+        <p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
     </div>
 </div>
+<div style="text-align:center; margin-top: 0em; margin-bottom: 0em"><p style="margin-top: 0.25em; margin-bottom: 0em;">TheBloke's LLM work is generously supported by a grant from <a href="https://a16z.com">andreessen horowitz (a16z)</a></p></div>
+<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
 <!-- header end -->
 # Fire Balloon's Baichuan Vicuna 7B GPTQ
   * Parameters: Groupsize = 128. Act Order / desc_act = False.
 <!-- footer start -->
+<!-- 200823 -->
 ## Discord
 For further support, and discussions on these models and AI in general, join us at:
+[TheBloke AI's Discord server](https://discord.gg/theblokeai)
 ## Thanks, and how to contribute.
 * Patreon: https://patreon.com/TheBlokeAI
 * Ko-Fi: https://ko-fi.com/TheBlokeAI
+**Special thanks to**: Aemon Algiz.
+**Patreon special mentions**: Sam, theTransient, Jonathan Leane, Steven Wood, webtim, Johann-Peter Hartmann, Geoffrey Montalvo, Gabriel Tamborski, Willem Michiel, John Villwock, Derek Yates, Mesiah Bishop, Eugene Pentland, Pieter, Chadd, Stephen Murray, Daniel P. Andersen, terasurfer, Brandon Frisco, Thomas Belote, Sid, Nathan LeClaire, Magnesian, Alps Aficionado, Stanislav Ovsiannikov, Alex, Joseph William Delisle, Nikolai Manek, Michael Davis, Junyu Yang, K, J, Spencer Kim, Stefan Sabev, Olusegun Samson, transmissions 11, Michael Levine, Cory Kujawski, Rainer Wilmers, zynix, Kalila, Luke @flexchar, Ajan Kanaga, Mandus, vamX, Ai Maven, Mano Prime, Matthew Berman, subjectnull, Vitor Caleffi, Clay Pascal, biorpg, alfie_i, 阿明, Jeffrey Morgan, ya boyyy, Raymond Fosdick, knownsqashed, Olakabola, Leonard Tan, ReadyPlayerEmma, Enrico Ros, Dave, Talal Aujan, Illia Dulskyi, Sean Connelly, senxiiz, Artur Olbinski, Elle, Raven Klaugh, Fen Risland, Deep Realms, Imad Khwaja, Fred von Graf, Will Dee, usrbinkat, SuperWojo, Alexandros Triantafyllidis, Swaroop Kallakuri, Dan Guido, John Detwiler, Pedro Madruga, Iucharbius, Viktor Bowallius, Asp the Wyvern, Edmond Seymore, Trenton Dambrowitz, Space Cruiser, Spiking Neurons AB, Pyrater, LangChain4j, Tony Hughes, Kacper Wikieł, Rishabh Srivastava, David Ziegler, Luke Pendergrass, Andrey, Gabriel Puliatti, Lone Striker, Sebastain Graf, Pierre Kircher, Randy H, NimbleBox.ai, Vadim, danny, Deo Leter
 Thank you to all my generous patrons and donaters!
+And thank you again to a16z for their generous grant.
 <!-- footer end -->
 # Original model card: Fire Balloon's Baichuan Vicuna 7B
 ```
 Q: How can I improve my time management skills?
+A: There are many different ways to improve your time management skills. Here are a few strategies that you may find helpful:
+1 Prioritize your tasks: Make a list of all the tasks you need to complete and prioritize them based on importance and urgency. This will help you focus on the most important tasks and avoid getting bogged down by less critical items.
+2 Use a time management planner: A planner can help you organize your time by breaking down your tasks into smaller, more manageable chunks. This can help you stay focused and on track.
+3 Set clear boundaries: Setting clear boundaries between work time and personal time can help you avoid distractions and stay focused on your tasks. This may involve designating certain times of day or days of the week for work and other times for rest and leisure.
+4 Take care of your physical and mental health: Taking care of your physical and mental health can help you be more productive and focused. This may involve getting enough sleep, eating a healthy diet, and taking breaks to relax and recharge.
 Remember that improving your time management skills takes practice and patience. Don't be discouraged if you make mistakes or experience setbacks. Keep trying and you will eventually develop better time management habits that will help you be more productive and successful.
 ```
 ```
 Q: Can you help me write a formal email to a potential business partner proposing a joint venture?
+A: Sure! Here is a sample formal email that you could use as a template to propose a joint venture with a
+potential business partner:
+Subject: Proposal for Joint Venture
+Dear [Business Partner],
+I hope this email finds you well. I am writing to propose a joint venture between our companies. We believe that a collaboration between our companies could bring significant benefits to both of us, and we would like to explore the possibility of forming a partnership.
+Our companies have complementary strengths, and we believe that a joint venture could allow us to leverage those strengths to create new products, services, and opportunities. We believe that a joint venture could also provide increased revenue and growth for both of our companies.
+In order to explore the possibility of a joint venture, we would like to schedule a call to discuss the details of our proposal. During the call, we would like to discuss the terms of the joint venture, including the specific roles and responsibilities of each partner, the financial terms of the agreement, and any other relevant details.
+I would be happy to schedule a call with you at your convenience. In the meantime, please feel free to contact me with any questions or concerns you may have about our proposal.
+Sincerely,
 [Your Name]
 ```
 ```
 Q: Implement a binary search algorithm to find a specific element in a sorted array.
+A: Here is an implementation of the binary search algorithm to find a specific element in a sorted array:
+ public static int binarySearch(int[] arr, int target) {
+     int left = 0;
+     int right = arr.length - 1;
+     while (left <= right) {
+         int mid = left + (right - left) / 2;
+         if (arr[mid] < target) {
+             left = mid + 1;
+         } else if (arr[mid] > target) {
+             right = mid - 1;
+         } else {
+             return mid;
+         }
+     }
+     return -1; // target not found
+ }
+Here is how the binary search algorithm works:
+ 1 Initialize two pointers, left and right, to the beginning and end of the array, respectively.
+ 2 While the left pointer is less than or equal to the right pointer, perform the following steps:
+    1 Calculate the middle index, mid, by dividing the left pointer and the right pointer.
+    2 Check if the middle element, arr[mid], is less than or equal to the target value.
+    3 If the middle element is less than the target value, update the left pointer to mid + 1.
+    4 If the middle element is greater than the target value, update the right pointer to mid - 1.
+    5 If the middle element is equal to the target value, the target element has been found, so return the middle index.
+ 3 If the target element is not found, return -1.
+This algorithm has a runtime complexity of O(log n) and a space complexity of O(1), making it an efficient way to find a specific element in a sorted array.
 ```
 ---
 ```
 Q: 晚上睡不着应该怎么办
+A:失眠是一种症状，而不是一种疾病。治疗失眠的目标是解决其潜在原因。失眠的一些常见原因包括：
+*睡眠不足
+*压力
+*心理健康问题 (焦虑，抑郁等)
+*药物 (处方和非处方)
+*环境因素 (噪音，光线，温度等)
+以下是一些通常可以帮助您睡眠的方法：
+*睡觉前4-6个小时避免刺激物 (咖啡因，尼古丁，酒精)
+*睡前冥想以减少压力和焦虑
+*睡觉前不要看亮屏幕 (电脑，手机，平板电脑等)
+*睡觉前至少一个小时不要运动
+*睡觉前不要吃食物
+*在睡觉前放松并平静下来 (看看书，喝杯牛奶等)
+如果这些技术不起作用，或者如果失眠是由药物或其他医学状况引起的，您应该联系您的医生。
 ```

config.json CHANGED Viewed

@@ -1,24 +1,34 @@
 {
-    "_name_or_path": "models/baichuan-vicuna-7b",
-    "architectures": [
-        "LlamaForCausalLM"
-    ],
-    "bos_token_id": 1,
-    "eos_token_id": 2,
-    "hidden_act": "silu",
-    "hidden_size": 4096,
-    "initializer_range": 0.02,
-    "intermediate_size": 11008,
-    "max_position_embeddings": 4096,
-    "max_sequence_length": 4096,
-    "model_type": "llama",
-    "num_attention_heads": 32,
-    "num_hidden_layers": 32,
-    "pad_token_id": 0,
-    "rms_norm_eps": 1e-06,
-    "tie_word_embeddings": false,
-    "torch_dtype": "float16",
-    "transformers_version": "4.29.2",
-    "use_cache": true,
-    "vocab_size": 64000
 }

 {
+  "_name_or_path": "models/baichuan-vicuna-7b",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "max_sequence_length": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "pad_token_id": 0,
+  "rms_norm_eps": 1e-06,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.29.2",
+  "use_cache": true,
+  "vocab_size": 64000,
+  "quantization_config": {
+    "bits": 4,
+    "group_size": 128,
+    "damp_percent": 0.01,
+    "desc_act": false,
+    "sym": true,
+    "true_sequential": true,
+    "model_file_base_name": "model",
+    "quant_method": "gptq"
+  }
 }

baichuan-vicuna-7b-GPTQ-4bit-128g.no-act.order.safetensors → model.safetensors RENAMED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d6ca522794a4da3df942618260652196b2cbe9999e2ff5de506ce682bb956041
-size 4520341352

 version https://git-lfs.github.com/spec/v1
+oid sha256:e9da49a5138bdd36012f4a97caa29226b2f2486963fd95d0f0966d18308290d2
+size 4520341408

quantize_config.json CHANGED Viewed

@@ -1,8 +1,9 @@
 {
-    "bits": 4,
-    "group_size": 128,
-    "damp_percent": 0.01,
-    "desc_act": false,
-    "sym": true,
-    "true_sequential": true
 }

 {
+  "bits": 4,
+  "group_size": 128,
+  "damp_percent": 0.01,
+  "desc_act": false,
+  "sym": true,
+  "true_sequential": true,
+  "model_file_base_name": "model"
 }