Duplicate from AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct
Browse filesCo-authored-by: Ariel Ekgren <Ekgren@users.noreply.huggingface.co>
- .gitattributes +34 -0
- GPT-SW3_MODEL_GUIDELINES +47 -0
- LICENSE +172 -0
- README.md +288 -0
- config.json +37 -0
- model-00001-of-00003.safetensors +3 -0
- model-00002-of-00003.safetensors +3 -0
- model-00003-of-00003.safetensors +3 -0
- model.safetensors.index.json +460 -0
- pytorch_model-00001-of-00003.bin +3 -0
- pytorch_model-00002-of-00003.bin +3 -0
- pytorch_model-00003-of-00003.bin +3 -0
- pytorch_model.bin.index.json +460 -0
- spiece.model +3 -0
- tokenizer_config.json +5 -0
.gitattributes
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
GPT-SW3_MODEL_GUIDELINES
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
GPT-SW3 Model Guidelines and Use Policies
|
| 2 |
+
|
| 3 |
+
Introduction
|
| 4 |
+
|
| 5 |
+
To ensure the responsible and sustainable use of the GPT-SW3 Model, we have established these guidelines and use policies. Every user must adhere to them to guarantee the shared objective of open and fair AI application.
|
| 6 |
+
|
| 7 |
+
1. Acceptable Use
|
| 8 |
+
|
| 9 |
+
Users must ensure the application of the AI model respects the principles of fairness, inclusivity, and responsible standards.
|
| 10 |
+
|
| 11 |
+
Prohibited Activities:
|
| 12 |
+
|
| 13 |
+
Any activities not in line with Lawful AI or Lawful AI-use as specified from time to time in the Ethics Guidelines for Trustworthy Artificial Intelligence created by High Level Expert Group on AI[1].
|
| 14 |
+
|
| 15 |
+
Regardless, engaging in illegal or harmful activities, including terrorism, child exploitation, and human trafficking, discrimination, harassment, or harm based on race, gender, ethnicity, or any other protected category, unauthorized practice of regulated professions: such as medical, financial, or legal without proper qualifications, infringing on third-party rights, generating malicious code, or harming infrastructure shall never be other than prohibited activities.
|
| 16 |
+
|
| 17 |
+
2. Redistribution & Attribution
|
| 18 |
+
|
| 19 |
+
Users can redistribute the Model or derivatives but must always credit the Model. When redistributing, these guidelines shall be shared and measures shall be taken to ensure the next user abides by them.
|
| 20 |
+
|
| 21 |
+
3. Feedback and Continuous Improvement
|
| 22 |
+
|
| 23 |
+
Users are encouraged to provide feedback on the Model's performance and any potential ethical and/or harmful issues . Users are encouraged to suggest enhancements that promote inclusivity, fairness, transparency, and utility.
|
| 24 |
+
|
| 25 |
+
4. Handling Misinformation and Misrepresentation
|
| 26 |
+
|
| 27 |
+
Do not use the model to promote, or spread misinformation. Never represent AI-generated outputs as human-generated content.
|
| 28 |
+
|
| 29 |
+
5. Personal and Sensitive Data
|
| 30 |
+
|
| 31 |
+
Never use the Model to process or generate personal or sensitive information without explicit consent and legal rights. Always respect privacy rights and data protection laws when using the model.
|
| 32 |
+
|
| 33 |
+
6. Engagement with the AI Community
|
| 34 |
+
|
| 35 |
+
Users are encouraged to collaborate with peers, sharing knowledge and best practices related to the Model. Consider joining or initiating forums, workshops, or discussions to promote the responsible use of the Model.
|
| 36 |
+
|
| 37 |
+
7. Monitoring and Compliance
|
| 38 |
+
|
| 39 |
+
Regularly review these guidelines and policies to ensure continuous compliance. Understand that non-compliance might lead to the revocation of the license.
|
| 40 |
+
|
| 41 |
+
8. Dispute Resolution
|
| 42 |
+
|
| 43 |
+
Approach Lindholmen Science Park directly in case of any conflicts or concerns related to the Model. Both parties should prioritize amicable resolution methods before considering legal action. Every user of the Model, whether an individual, organization, or developer, has the responsibility to prioritize the responsible use of AI. By adhering to these guidelines and use policies, we can collaboratively ensure that AI serves as a tool for progress, fairness, and societal enhancement.
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
________________
|
| 47 |
+
[1] https://digital-strategy.ec.europa.eu/en/policies/expert-group-ai
|
LICENSE
ADDED
|
@@ -0,0 +1,172 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
AI Sweden's LLM AI Model License Agreement
|
| 2 |
+
|
| 3 |
+
1. Introduction
|
| 4 |
+
|
| 5 |
+
This license governs the use of GPT-SW3 (the "Model"). By accessing or using the Model, you agree to adhere to these terms. This license aims to promote the open and responsible use of our AI while ensuring the security and respect of users' rights.
|
| 6 |
+
|
| 7 |
+
2. Grant of License
|
| 8 |
+
|
| 9 |
+
Lindholmen Science Park hereby grants you, under the terms and conditions stipulated in the AI Sweden’s LLM AI Model License Agreement, (the “Agreement”), a non-exclusive, transferable, worldwide, and royalty-free license to:
|
| 10 |
+
* Use: Access and utilize the Model in accordance with the set limitations in section 7. Acceptable Use Policy and within the stipulated guidelines.
|
| 11 |
+
* Modify: Adjust, adapt, and build upon the Model to better suit your individual or organizational needs, while still adhering to the principles and guidelines established herein.
|
| 12 |
+
* Distribute: Share the Model and your derivative works with others, provided such distribution is in compliance with the terms of this license.
|
| 13 |
+
|
| 14 |
+
Nature of the Grant
|
| 15 |
+
* Non-exclusive: The license does not confer any form of exclusivity; Lindholmen Science Park reserves the right to grant similar licenses to other individuals or entities.
|
| 16 |
+
* Transferable: You are permitted to sub-license, rent, lease, assign, or otherwise transfer your rights under this license to any third party.
|
| 17 |
+
* Worldwide: This license permits the use, modification, and distribution of the Model anywhere globally, encouraging a collaborative, international approach to furthering the Model's development and application.
|
| 18 |
+
* Royalty-free: You are not required to pay any form of royalties to Lindholmen Science Park for the rights granted under this license, fostering open and unrestricted access to the Model.
|
| 19 |
+
|
| 20 |
+
Obligations of the Licensee
|
| 21 |
+
* Compliance: You must comply with all the terms and conditions set forth in this license to maintain the rights granted herein.
|
| 22 |
+
|
| 23 |
+
Reservation of Rights
|
| 24 |
+
* Intellectual Property: This license does not transfer any ownership of the intellectual property associated with the Model. Lindholmen Science Park retains all intellectual property rights not expressly granted in this license.
|
| 25 |
+
* Amendments: Lindholmen Science Park reserves the right to alter the terms of this license in the future, in response to changing technological and societal landscapes, ensuring a dynamic and adaptive licensing approach.
|
| 26 |
+
|
| 27 |
+
3. Scope of Use
|
| 28 |
+
|
| 29 |
+
Non-commercial Use
|
| 30 |
+
* Education and Research: Licensees are encouraged to use the Model for educational and research purposes, fostering knowledge sharing and innovation.
|
| 31 |
+
* Public Service: Licensees can leverage the Model to build applications and services aimed at community welfare and public good.
|
| 32 |
+
|
| 33 |
+
Commercial Use
|
| 34 |
+
* Business Ventures: Licensees are permitted to use the Model to create products, services, or applications that seek to generate profit, encouraging entrepreneurial initiatives.
|
| 35 |
+
* Corporate Research: Licensees may employ the Model for corporate research, enhancing business strategies and solutions.
|
| 36 |
+
|
| 37 |
+
4. Attribution Requirements
|
| 38 |
+
|
| 39 |
+
Non-commercial Use
|
| 40 |
+
* Acknowledgement: While it is encouraged, non-commercial users are not strictly required to attribute the Model; however, voluntary attribution is appreciated to acknowledge the efforts of the original creators.
|
| 41 |
+
|
| 42 |
+
Commercial Use
|
| 43 |
+
* Mandatory Attribution: Commercial users are mandated to provide clear and conspicuous attribution to the original creators of the Model, promoting transparency and credit where it's due.
|
| 44 |
+
|
| 45 |
+
Format of Attribution The original creators are AI Sweden, RISE and WASP. The attribution should be presented in a manner that is reasonable and customarily used in commercial products or services, which could include but is not limited to:
|
| 46 |
+
* Documentation: Including attribution in the user manuals, installation guides, or on the official website where the product or service is detailed.
|
| 47 |
+
* Application Interface: Incorporating attribution in a dedicated "About" or "Credits" section within the application or service interface.
|
| 48 |
+
* Marketing Materials: Featuring attribution in marketing and promotional materials, highlighting the utilization of the Model in the commercial offering.
|
| 49 |
+
|
| 50 |
+
5. Redistribution
|
| 51 |
+
|
| 52 |
+
* Redistributor: A person or entity that shares the Model with a third party, either in its original form or with modifications.
|
| 53 |
+
|
| 54 |
+
Authorization for Redistribution
|
| 55 |
+
* Original Model: Licensees are permitted to redistribute the Model in its original form, provided that they comply with the terms and conditions outlined in this license.
|
| 56 |
+
* Modified Model: Licensees are permitted to modify and redistribute the Model, inclusive of derivative works created through the modification of the original Model, aligning with the stipulations delineated in this license.
|
| 57 |
+
|
| 58 |
+
Adherence to the License
|
| 59 |
+
* Binding Effect on Derivative Works: Any derivative work based on the Model shall be governed by the terms and conditions of this license, ensuring a consistent ethical and legal framework for all adaptations of the Model.
|
| 60 |
+
|
| 61 |
+
Transparency and Documentation
|
| 62 |
+
* Modifications: Redistributors are required to clearly indicate the nature and extent of modifications undertaken, fostering transparency and informed use.
|
| 63 |
+
|
| 64 |
+
No Misrepresentation
|
| 65 |
+
* Original Endorsement: Redistributors must not convey or imply any endorsement by the original creators of the Model for the redistributed or derivative work, avoiding any misrepresentation or undue association.
|
| 66 |
+
|
| 67 |
+
Safety and Ethical Considerations
|
| 68 |
+
* Obligation to Maintain Standards: Redistributors must ensure that the Model, whether in its original or modified form, maintains a standard of safety, ethical utility, and respect for users’ rights, consistent with the objectives and principles embodied in this license.
|
| 69 |
+
|
| 70 |
+
Regulatory Compliance
|
| 71 |
+
* Legal Adherence: Redistributors are mandated to ensure compliance with applicable legal and regulatory norms, including data protection laws, while redistributing the Model or any derivative works.
|
| 72 |
+
|
| 73 |
+
Feedback and Community Engagement
|
| 74 |
+
* Feedback Channels: Redistributors are encouraged to establish feedback channels for users and stakeholders, facilitating a collaborative approach towards the continual improvement of the Model and derivative works.
|
| 75 |
+
|
| 76 |
+
6. Third-party Integrations
|
| 77 |
+
|
| 78 |
+
Compliance with License Terms: Third parties who integrate the Model into their own systems, products, or services ("Integrated Products") must ensure that such Integrated Products are in full compliance with the terms and conditions outlined in this license. This entails a commitment to uphold the ethical, responsible and lawful utilization of the Model.
|
| 79 |
+
|
| 80 |
+
Notification and Transparency: Third parties are required to clearly notify users of Integrated Products about the incorporation of the Model and ensure that the terms of this license are made accessible to the users to foster transparency and informed usage.
|
| 81 |
+
|
| 82 |
+
Liability: Third parties are responsible for any liabilities arising from their non-compliance with the terms of this license in relation to the use of the Model in Integrated Products. They are required to indemnify Lindholmen Science Park against claims, damages, and losses arising out of such non-compliance.
|
| 83 |
+
|
| 84 |
+
Security and Privacy: Third parties must undertake necessary measures to ensure the security and privacy of the users’ data while using Integrated Products, including adherence to applicable data protection laws and regulations.
|
| 85 |
+
|
| 86 |
+
Intellectual Property Rights: The integration of the Model into third-party systems must respect and preserve the intellectual property rights of Lindholmen Science Park concerning the Model, including trademarks, copyrights, and patents, as applicable.
|
| 87 |
+
|
| 88 |
+
Termination: In case of violation of any terms of this license by third parties, Lindholmen Science Park reserves the right to terminate the license granted to such third parties, requiring them to cease the use and integration of the Model in their systems immediately.
|
| 89 |
+
|
| 90 |
+
Feedback and Cooperation: Third parties are encouraged to maintain a collaborative relationship with Lindholmen Science Park, providing feedback on the Model’s performance in Integrated Products and cooperating in efforts to enhance the Model's functionalities and rectify any issues.
|
| 91 |
+
|
| 92 |
+
7. Acceptable Use Policy
|
| 93 |
+
|
| 94 |
+
Definition of Guidelines and Use Policies: The "GPT-SW3 Model Guidelines and Use Policies" refer to the structured set of principles, rules, and parameters established by Lindholmen Science Park which govern the acceptable use of the Model. These policies are devised to foster safe, ethical, responsible and sustainable use of the Model that guarantees the shared objective of open and fair Ai application.
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
Compliance Obligation: By accessing or utilizing the Model, you expressly agree to adhere to the GPT-SW3 Guidelines and Use Policies as laid down by Lindholmen Science Park. This entails using the Model responsibly, and lawfully, in accordance with the stipulated guidelines.
|
| 98 |
+
|
| 99 |
+
Updates and Modifications: Lindholmen Science Park reserves the right to periodically review, amend, or update the GPT-SW3 Guidelines and Use Policies to adapt to technological advancements, legal developments, or societal changes. Users are responsible for keeping themselves abreast of the latest updates to ensure ongoing compliance.
|
| 100 |
+
|
| 101 |
+
Feedback and Reporting Mechanism: Users are encouraged to actively engage with Lindholmen Science Park through designated channels to report any misuse, violations, or to suggest improvements regarding the Model's functioning, thus fostering a collaborative environment for the enhancement of the Model.
|
| 102 |
+
|
| 103 |
+
Consequences of Violation: Non-compliance with the GPT-SW3 Guidelines and Use Policies may result in punitive actions, including but not limited to, the temporary or permanent revocation of access rights to the Model, legal actions, and/or public disclosure of the violation, as deemed appropriate by Lindholmen Science Park.
|
| 104 |
+
|
| 105 |
+
Indemnification: Users agree to indemnify and hold harmless Lindholmen Science Park against any claims, damages, or liabilities arising out of the violation of the GPT-SW3 Guidelines and Use Policies.
|
| 106 |
+
|
| 107 |
+
8. Feedback
|
| 108 |
+
|
| 109 |
+
Commitment to Continuous Improvement: Lindholmen Science Park values the insights and perspectives of our users. We are committed to continuous improvement and encourage users to actively participate in the evolution of the Model through constructive feedback, reporting of issues, and suggestions for enhancements.
|
| 110 |
+
|
| 111 |
+
Feedback Channels: Users may provide their feedback through the following channels:
|
| 112 |
+
|
| 113 |
+
Email: Reach us at nlu@ai.se
|
| 114 |
+
Community Forums: Join discussions and share your insights on our community forums at AI Nordics Discord - http://discord.gg/RgKVztg3xU
|
| 115 |
+
|
| 116 |
+
Anonymous Feedback: We welcome anonymous feedback to allow users to share their perspectives freely. However, we encourage users to provide contact information to facilitate follow-up discussions and updates on the addressed issues.
|
| 117 |
+
|
| 118 |
+
Responsiveness: While we endeavor to review and consider all feedback received, we cannot guarantee a response to every submission. We appreciate your understanding and patience as we work diligently to enhance the Model for all users.
|
| 119 |
+
|
| 120 |
+
Intellectual Property: By submitting feedback, you grant Lindholmen Science Park a worldwide, non-exclusive, royalty-free, perpetual, irrevocable license to use, reproduce, modify, adapt, publish, distribute, and incorporate such feedback into our work and research, without acknowledgement or compensation to you.
|
| 121 |
+
|
| 122 |
+
9. Warranty and Liability
|
| 123 |
+
|
| 124 |
+
The model is provided "as is," and Lindholmen Science Park disclaims all warranties, express or implied.
|
| 125 |
+
No Warranty: THE MODEL IS PROVIDED "AS IS," WITHOUT WARRANTY OF ANY KIND, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF PERFORMANCE, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, ACCURACY, OMISSIONS, COMPLETENESS, CURRENTNESS, AND DELAYS.
|
| 126 |
+
|
| 127 |
+
Cap on Liability: TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, LINDHOLMEN SCIENCE PARK 'S CUMULATIVE LIABILITY TO YOU, FOR ANY AND ALL CLAIMS RELATED TO THE MODEL, SHALL NOT EXCEED AN AGGREGATE AMOUNT EQUAL TO THE LESSER OF (i) € 500 OR (ii) THE TOTAL AMOUNTS YOU PAID TO LINDHOLMEN SCIENCE PARK IN THE TWELVE (12) MONTHS IMMEDIATELY PRECEDING THE INCIDENT GIVING RISE TO THE LIABILITY.
|
| 128 |
+
|
| 129 |
+
Exclusion of Certain Liabilities: LINDHOLMEN SCIENCE PARK SHALL NOT BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, PUNITIVE, CONSEQUENTIAL, OR ANY OTHER FORM OF DAMAGES, INCLUDING, BUT NOT LIMITED TO, LOSS OF PROFITS, DATA, GOODWILL, OR ANY OTHER INTANGIBLE LOSSES, ARISING OUT OF OR RELATED TO THIS AGREEMENT, THE USE OR THE INABILITY TO USE THE MODEL, EVEN IF LINDHOLMEN SCIENCE PARK HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
|
| 130 |
+
|
| 131 |
+
Essential Purpose: THE LIMITATIONS SPECIFIED IN THIS SECTION SHALL SURVIVE AND APPLY EVEN IF ANY LIMITED REMEDY SPECIFIED IN THIS AGREEMENT IS FOUND TO HAVE FAILED ITS ESSENTIAL PURPOSE.
|
| 132 |
+
|
| 133 |
+
Basis of the Bargain: THE LIMITATIONS OF DAMAGES SET FORTH ABOVE ARE FUNDAMENTAL ELEMENTS OF THE BASIS OF THE BARGAIN BETWEEN LINDHOLMEN SCIENCE PARK AND YOU.
|
| 134 |
+
|
| 135 |
+
Local Laws and Regulations: Nothing in this clause aims to limit or exclude any liability that cannot be limited or excluded under applicable laws. Users are encouraged to be aware of and adhere to local laws and regulations governing the use of AI models and services.
|
| 136 |
+
|
| 137 |
+
10. Termination Clauses
|
| 138 |
+
|
| 139 |
+
* Model Materials: Refers to the Model and all associated documentation, guidelines, and policies provided by Lindholmen Science Park.
|
| 140 |
+
Commencement and Duration
|
| 141 |
+
|
| 142 |
+
* This Agreement shall become effective upon your acceptance or when you commence access to or use of the Model Materials (“Effective Date”) and shall remain in full force and effect unless earlier terminated in accordance with this Agreement.
|
| 143 |
+
Termination by Organization
|
| 144 |
+
|
| 145 |
+
* Breach: Lindholmen Science Park reserves the right to terminate this Agreement unilaterally if you are found to be in breach of any term or condition stipulated in this Agreement.
|
| 146 |
+
|
| 147 |
+
* Safety and Compliance: Lindholmen Science Park further reserves the right to terminate this Agreement to comply with any applicable law, regulation, or guideline, or to preserve the safety, integrity, and lawful operation of the Model and associated resources.
|
| 148 |
+
Termination by You
|
| 149 |
+
|
| 150 |
+
* You have the right to terminate this Agreement at any time by ceasing all use of the Model Materials and deleting all copies of the Materials in your possession or control.
|
| 151 |
+
Consequences of Termination
|
| 152 |
+
|
| 153 |
+
* Ceasing Use: Upon termination of this Agreement for any reason the license is also terminated. Therefore, if the Agreement is terminated you must cease all use of the Model Materials and promptly delete and destroy all copies, full or partial, of the Materials in your possession or control.
|
| 154 |
+
|
| 155 |
+
* Survival of Rights and Obligations: The rights and obligations contained in Sections 6 and 8 of this Agreement shall survive the termination of this Agreement and shall continue to bind you and any permitted successors and assignees.
|
| 156 |
+
|
| 157 |
+
* No Liability for Termination: Lindholmen Science Park will not be liable for any damages, losses, costs, or harms arising from the termination of this Agreement, and termination will not affect any liability accrued before the termination date.
|
| 158 |
+
|
| 159 |
+
11. Jurisdiction and Governing Law
|
| 160 |
+
|
| 161 |
+
Any dispute, controversy, or claim arising out of or in connection with this contract, or the breach, termination, or invalidity thereof, shall be finally settled by arbitration administered by the Stockholm Chamber of Commerce Arbitration Institute (the “SCC”).
|
| 162 |
+
The Rules for Expedited Arbitrations shall apply, unless the SCC in its discretion determines, taking into account the complexity of the case, the amount in dispute and other circumstances, that the Arbitration Rules shall apply. In the latter case, the SCC shall also decide whether the Arbitral Tribunal shall be composed of one or three arbitrators. The seat of arbitration shall be Stockholm, Sweden. The language to be used in the arbitral proceedings shall be English. This contract shall be governed by the substantive law of Sweden.
|
| 163 |
+
|
| 164 |
+
12. Updates and Revisions
|
| 165 |
+
|
| 166 |
+
Commitment to Update: Lindholmen Science Park acknowledges the rapid pace of technological and societal advancements. We remain committed to periodically reviewing and updating the terms of this Agreement to remain in harmony with such developments, thereby safeguarding the interests of all stakeholders involved while promoting sustainable and responsible AI use.
|
| 167 |
+
|
| 168 |
+
Right to Amend: Lindholmen Science Park reserves the right, at its sole discretion, to amend, modify, or replace any part of this Agreement. It is your responsibility to check this Agreement periodically for changes. Your continued use of or access to the Model Materials following the posting of any changes to this Agreement constitutes acceptance of those changes.
|
| 169 |
+
|
| 170 |
+
Grace Period: In the event of any substantial amendment to the terms of this Agreement, you will be provided with a notice period of 30 days from the date such amendments are posted for you to review through the distribution platform to review and adapt to the amended terms. Should you disagree with the amendments, you reserve the right to terminate this Agreement in accordance with the termination clause herein.
|
| 171 |
+
|
| 172 |
+
Adherence to Future Norms: By agreeing to this Agreement, you commit to adhering to potential future norms, regulations, and guidelines that may be introduced in the jurisdiction pertaining to the use of AI technologies, even if they are introduced after your acceptance of this Agreement.
|
README.md
ADDED
|
@@ -0,0 +1,288 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
datasets:
|
| 4 |
+
- laion/OIG
|
| 5 |
+
- databricks/databricks-dolly-15k
|
| 6 |
+
- OpenAssistant/oasst1
|
| 7 |
+
language:
|
| 8 |
+
- da
|
| 9 |
+
- sv
|
| 10 |
+
- 'no'
|
| 11 |
+
- en
|
| 12 |
+
- is
|
| 13 |
+
pipeline_tag: conversational
|
| 14 |
+
---
|
| 15 |
+
# Model description
|
| 16 |
+
[AI Sweden](https://huggingface.co/AI-Sweden-Models/)
|
| 17 |
+
**Base models**
|
| 18 |
+
[GPT-Sw3 126M](https://huggingface.co/AI-Sweden-Models/gpt-sw3-126m/) | [GPT-Sw3 356M](https://huggingface.co/AI-Sweden-Models/gpt-sw3-356m/) | [GPT-Sw3 1.3B](https://huggingface.co/AI-Sweden-Models/gpt-sw3-1.3b/)
|
| 19 |
+
[GPT-Sw3 6.7B](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b/) | [GPT-Sw3 6.7B v2](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b-v2/) | [GPT-Sw3 20B](https://huggingface.co/AI-Sweden-Models/gpt-sw3-20b/)
|
| 20 |
+
[GPT-Sw3 40B](https://huggingface.co/AI-Sweden-Models/gpt-sw3-40b/)
|
| 21 |
+
**Instruct models**
|
| 22 |
+
[GPT-Sw3 126M Instruct](https://huggingface.co/AI-Sweden-Models/gpt-sw3-126m-instruct/) | [GPT-Sw3 356M Instruct](https://huggingface.co/AI-Sweden-Models/gpt-sw3-356m-instruct/) | [GPT-Sw3 1.3B Instruct](https://huggingface.co/AI-Sweden-Models/gpt-sw3-1.3b-instruct/)
|
| 23 |
+
[GPT-Sw3 6.7B v2 Instruct](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct/) | [GPT-Sw3 20B Instruct](https://huggingface.co/AI-Sweden-Models/gpt-sw3-20b-instruct/)
|
| 24 |
+
**Quantized models**
|
| 25 |
+
[GPT-Sw3 6.7B v2 Instruct 4-bit gptq](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct-4bit-gptq) | [GPT-Sw3 20B Instruct 4-bit gptq](https://huggingface.co/AI-Sweden-Models/gpt-sw3-20b-instruct-4bit-gptq)
|
| 26 |
+
|
| 27 |
+
GPT-SW3 is a collection of large decoder-only pretrained transformer language models that were developed by AI Sweden in collaboration with RISE and the WASP WARA for Media and Language. GPT-SW3 has been trained on a dataset containing 320B tokens in Swedish, Norwegian, Danish, Icelandic, English, and programming code. The model was pretrained using a causal language modeling (CLM) objective utilizing the NeMo Megatron GPT implementation.
|
| 28 |
+
|
| 29 |
+
The `instruct` models were finetrained on instruction data using both chat and raw text formats.
|
| 30 |
+
|
| 31 |
+
# Intended use
|
| 32 |
+
GPT-SW3 is an autoregressive large language model that is capable of generating coherent text in 5 different languages, and 4 programming languages. GPT-SW3 can also be instructed to perform text tasks that it has not been explicitly trained for, by casting them as text generation tasks.
|
| 33 |
+
|
| 34 |
+
# Limitations
|
| 35 |
+
Like other large language models for which the diversity (or lack thereof) of training data induces downstream impact on the quality of our model, GPT-SW3 has limitations in terms of for example bias and safety. GPT-SW3 can also have quality issues in terms of generation diversity and hallucination. By releasing with the modified RAIL license, we also hope to increase communication, transparency, and the study of large language models. The model may: overrepresent some viewpoints and underrepresent others, contain stereotypes, generate hateful, abusive, violent, discriminatory or prejudicial language. The model may make errors, including producing incorrect information as if it were factual, it may generate irrelevant or repetitive outputs, and content that may not be appropriate for all settings, including sexual content.
|
| 36 |
+
|
| 37 |
+
# How to use
|
| 38 |
+
To be able to access the model from Python, since this is a private repository, you have to log in with your access token. This can be done with `huggingface-cli login`, see [HuggingFace Quick Start Guide](https://huggingface.co/docs/huggingface_hub/quick-start#login) for more information.
|
| 39 |
+
|
| 40 |
+
The following code snippet loads our tokenizer & model, and uses the GPU if available.
|
| 41 |
+
|
| 42 |
+
```python
|
| 43 |
+
import torch
|
| 44 |
+
from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
|
| 45 |
+
|
| 46 |
+
# Initialize Variables
|
| 47 |
+
model_name = "AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct"
|
| 48 |
+
device = "cuda:0" if torch.cuda.is_available() else "cpu"
|
| 49 |
+
prompt = "Träd är fina för att"
|
| 50 |
+
|
| 51 |
+
# Initialize Tokenizer & Model
|
| 52 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 53 |
+
model = AutoModelForCausalLM.from_pretrained(model_name)
|
| 54 |
+
model.eval()
|
| 55 |
+
model.to(device)
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
Generating text using the `generate` method is done as follows:
|
| 59 |
+
```python
|
| 60 |
+
input_ids = tokenizer(prompt, return_tensors="pt")["input_ids"].to(device)
|
| 61 |
+
|
| 62 |
+
generated_token_ids = model.generate(
|
| 63 |
+
inputs=input_ids,
|
| 64 |
+
max_new_tokens=100,
|
| 65 |
+
do_sample=True,
|
| 66 |
+
temperature=0.6,
|
| 67 |
+
top_p=1,
|
| 68 |
+
)[0]
|
| 69 |
+
|
| 70 |
+
generated_text = tokenizer.decode(generated_token_ids)
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
The chat format used during data-preprocessing takes the form:
|
| 74 |
+
```
|
| 75 |
+
<|endoftext|><s>
|
| 76 |
+
User:
|
| 77 |
+
Jag tycker träd är fina
|
| 78 |
+
<s>
|
| 79 |
+
Bot:
|
| 80 |
+
Kul att du tycker det!
|
| 81 |
+
<s>
|
| 82 |
+
...
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
The procedure to generate text is the same as before:
|
| 86 |
+
|
| 87 |
+
```python
|
| 88 |
+
prompt = """
|
| 89 |
+
<|endoftext|><s>
|
| 90 |
+
User:
|
| 91 |
+
Varför är träd fina?
|
| 92 |
+
<s>
|
| 93 |
+
Bot:
|
| 94 |
+
""".strip()
|
| 95 |
+
|
| 96 |
+
input_ids = tokenizer(prompt, return_tensors="pt")["input_ids"].to(device)
|
| 97 |
+
|
| 98 |
+
generated_token_ids = model.generate(
|
| 99 |
+
inputs=input_ids,
|
| 100 |
+
max_new_tokens=100,
|
| 101 |
+
do_sample=True,
|
| 102 |
+
temperature=0.6,
|
| 103 |
+
top_p=1,
|
| 104 |
+
)[0]
|
| 105 |
+
|
| 106 |
+
generated_text = tokenizer.decode(generated_token_ids)
|
| 107 |
+
```
|
| 108 |
+
|
| 109 |
+
Generating text using the `generate` method is done as follows:
|
| 110 |
+
```python
|
| 111 |
+
input_ids = tokenizer(prompt, return_tensors="pt")["input_ids"].to(device)
|
| 112 |
+
|
| 113 |
+
generated_token_ids = model.generate(
|
| 114 |
+
inputs=input_ids,
|
| 115 |
+
max_new_tokens=100,
|
| 116 |
+
do_sample=True,
|
| 117 |
+
temperature=0.6,
|
| 118 |
+
top_p=1,
|
| 119 |
+
)[0]
|
| 120 |
+
|
| 121 |
+
A convenient alternative to the `generate` method is the HuggingFace pipeline, which handles most of the work for you:
|
| 122 |
+
```python
|
| 123 |
+
generator = pipeline('text-generation', tokenizer=tokenizer, model=model, device=device)
|
| 124 |
+
generated = generator(prompt, max_new_tokens=100, do_sample=True, temperature=0.6, top_p=1)[0]["generated_text"]
|
| 125 |
+
```
|
| 126 |
+
|
| 127 |
+
# Compliance
|
| 128 |
+
The release of GPT-SW3 consists of model weights, a configuration file, a tokenizer file and a vocabulary file. None of these files contain any personally identifiable information (PII) or any copyrighted material.
|
| 129 |
+
|
| 130 |
+
# GPT-SW3 Model Card
|
| 131 |
+
Following Mitchell et al. (2018), we provide a model card for GPT-SW3.
|
| 132 |
+
|
| 133 |
+
# Model Details
|
| 134 |
+
- Person or organization developing model: GPT-SW3 was developed by AI Sweden in collaboration with RISE and the WASP WARA for Media and Language.
|
| 135 |
+
- Model date: GPT-SW3 date of release 2022-12-20
|
| 136 |
+
- Model version: This is the second generation of GPT-SW3.
|
| 137 |
+
- Model type: GPT-SW3 is a large decoder-only transformer language model.
|
| 138 |
+
- Information about training algorithms, parameters, fairness constraints or other applied approaches, and features: GPT-SW3 was trained with the NeMo Megatron GPT implementation.
|
| 139 |
+
- Paper or other resource for more information: N/A.
|
| 140 |
+
- License: [LICENSE](https://huggingface.co/AI-Sweden-Models/gpt-sw3-6.7b-v2-instruct/blob/main/LICENSE).
|
| 141 |
+
- Where to send questions or comments about the model: nlu@ai.se
|
| 142 |
+
|
| 143 |
+
# Intended Use
|
| 144 |
+
- Primary intended uses: We pre-release GPT-SW3 for research and evaluation of the capabilities of Large Language Models for the Nordic languages. This is an important step in the process of knowledge building for LLMs, validating the model and collecting feedback on both what works well and what does not.
|
| 145 |
+
- Primary intended users: Organizations and individuals in the Nordic NLP ecosystem who can contribute to the validation and testing of the models and provide feedback to the community.
|
| 146 |
+
- Out-of-scope use cases: See the modified RAIL license.
|
| 147 |
+
|
| 148 |
+
# Data, Limitations, and Recommendations
|
| 149 |
+
- Data selection for training: Training data for GPT-SW3 was selected based on a combination of breadth and availability. See our Datasheet for more detailed information on the data used to train our model.
|
| 150 |
+
- Data selection for evaluation: N/A
|
| 151 |
+
- Limitations: Like other large language models for which the diversity (or lack thereof) of training data induces downstream impact on the quality of our model, GPT-SW3 has limitations in terms of bias and safety. GPT-SW3 can also have quality issues in terms of generation diversity and hallucination. In general, GPT-SW3 is not immune from the plethora of issues that plague modern large language models. By releasing with the modified RAIL license, we also hope to increase communication, transparency, and the study of large language models. The model may: Overrepresent some viewpoints and underrepresent others. Contain stereotypes. Generate: Hateful, abusive, or violent language. Discriminatory or prejudicial language. Content that may not be appropriate for all settings, including sexual content. Make errors, including producing incorrect information as if it were factual. Generate irrelevant or repetitive outputs.
|
| 152 |
+
- Recommendations for future work: Indirect users should be made aware when the content they're working with is created by the LLM. Users should be aware of Risks and Limitations, and include an appropriate age disclaimer or blocking interface as necessary. Models pretrained with the LLM should include an updated Model Card. Users of the model should provide mechanisms for those affected to provide feedback, such as an email address for comments.
|
| 153 |
+
- We hope that the release of GPT-SW3, as well as information around our model training process, will increase open science around both large language models in specific and natural language processing and deep learning in general.
|
| 154 |
+
|
| 155 |
+
# GPT-SW3 Datasheet
|
| 156 |
+
- We follow the recommendations of Gebru et al. (2021) and provide a datasheet for the dataset used to train GPT-SW3.
|
| 157 |
+
|
| 158 |
+
# Motivation
|
| 159 |
+
- For what purpose was the dataset created? Was there a specific task in mind? Was there a specific gap that needed to be filled? Please provide a description. Pre-training of Large Language Models (LLM), such as GPT-3 (T. B. Brown et al., 2020), Gopher (J. W. Rae et al., 2022), BLOOM (T. L. Scao et al., 2022), etc. require 100s or even 1000s GBs of text data, with recent studies (Chinchilla: J. Hoffmann et al., 2022) suggesting that the scale of the training data is even more important than previously imagined. Therefore, in order to train Swedish LLMs, we needed a large scale Swedish dataset of high quality. Since no such datasets existed before this initiative, we collected data in the Nordic and English languages.
|
| 160 |
+
- Who created the dataset (e.g., which team, research group) and on behalf of which entity (e.g., company, institution, organization)? The Strategic Initiative Natural Language Understanding at AI Sweden has established a new research environment in which collaboration is key. The core team working on the creation of the dataset is the NLU research group at AI Sweden. This group consists of researchers and developers from AI Sweden (Lindholmen Science Park AB) and RISE.
|
| 161 |
+
- Who funded the creation of the dataset? If there is an associated grant, please provide the name of the grantor and the grant name and number. The Swedish Innovation Agency (Vinnova) has funded this work across several different grants, including 2019-02996 and 2022-00949.
|
| 162 |
+
- Any other comments? No.
|
| 163 |
+
|
| 164 |
+
# Composition
|
| 165 |
+
- What do the instances that comprise the dataset represent (e.g., documents, photos, people, countries)? Are there multiple types of instances (e.g., movies, users, and ratings; people and interactions between them; nodes and edges)? Please provide a description. The instances are textual documents categorized by language and document type. The dataset is a filtered and deduplicated collection that includes the following sources:
|
| 166 |
+
|
| 167 |
+
- Books
|
| 168 |
+
- Litteraturbanken (https://litteraturbanken.se/)
|
| 169 |
+
- The Pile
|
| 170 |
+
|
| 171 |
+
- Articles
|
| 172 |
+
- Diva (https://www.diva-portal.org/)
|
| 173 |
+
- The Pile: PubMed
|
| 174 |
+
- The Pile: ArXiv
|
| 175 |
+
|
| 176 |
+
- Code
|
| 177 |
+
- Code Parrot: Github code (https://huggingface.co/datasets/codeparrot/github-code)
|
| 178 |
+
|
| 179 |
+
- Conversational
|
| 180 |
+
- Familjeliv (https://www.familjeliv.se/)
|
| 181 |
+
- Flashback (https://flashback.se/)
|
| 182 |
+
- Datasets collected through Parlai (see Appendix in data paper for complete list) (https://github.com/facebookresearch/ParlAI)
|
| 183 |
+
- Pushshift.io Reddit dataset, developed in Baumgartner et al. (2020) and processed in Roller et al. (2021)
|
| 184 |
+
|
| 185 |
+
- Math
|
| 186 |
+
- English Math dataset generated with code from DeepMind (D. Saxton et al., 2019)
|
| 187 |
+
- Swedish Math dataset, generated as above with manually translated templates
|
| 188 |
+
|
| 189 |
+
- Miscellaneous
|
| 190 |
+
- Summarization data (https://www.ida.liu.se/~arnjo82/papers/clarin-21-julius.pdf)
|
| 191 |
+
- OPUS, the open parallel corpus (https://opus.nlpl.eu/)
|
| 192 |
+
- Movie scripts (https://github.com/Aveek-Saha/Movie-Script-Database)
|
| 193 |
+
- Natural Instructions (https://github.com/allenai/natural-instructions)
|
| 194 |
+
- P3 (Public Pool of Prompts), (https://huggingface.co/datasets/bigscience/P3)
|
| 195 |
+
- The Norwegian Colossal Corpus (https://huggingface.co/datasets/NbAiLab/NCC)
|
| 196 |
+
- Danish Gigaword (https://gigaword.dk/)
|
| 197 |
+
- Icelandic Gigaword (https://clarin.is/en/resources/gigaword/)
|
| 198 |
+
- The Pile: Stack Exchange
|
| 199 |
+
|
| 200 |
+
- Web Common Crawl
|
| 201 |
+
- Web data from the project LES (Linguistic Explorations of Societies, https://les.gu.se).
|
| 202 |
+
- Multilingual C4 (MC4), prepared by AllenAI from C4 (C. Raffel et al., 2019)
|
| 203 |
+
- Open Super-large Crawled Aggregated coRpus (OSCAR) (P. O. Suarez, 2019)
|
| 204 |
+
- The Pile: Open Web Text
|
| 205 |
+
|
| 206 |
+
- Web Sources
|
| 207 |
+
- Various public Swedish website scrapes (see Appendix in data paper)
|
| 208 |
+
- Familjeliv Articles
|
| 209 |
+
- Public Swedish Job Ads from JobTech/Arbetsförmedlingen
|
| 210 |
+
- Wikipedia
|
| 211 |
+
- Official Wikipedia dumps
|
| 212 |
+
|
| 213 |
+
- **Instruction data**:
|
| 214 |
+
- [dolly](https://github.com/databrickslabs/dolly/tree/master/data)
|
| 215 |
+
- [Open Assistant](https://github.com/LAION-AI/Open-Assistant/blob/main/docs/docs/data/datasets.md)
|
| 216 |
+
- [OIG](https://laion.ai/blog/oig-dataset/)
|
| 217 |
+
- Fass: Swedish pharmaceutical information, which was transformed into Q&A format.
|
| 218 |
+
|
| 219 |
+
- How many instances are there in total (of each type, if appropriate)? The training data consists of 1.1TB UTF-8 encoded text, containing 660M documents with a total of 320B tokens.
|
| 220 |
+
- Does the dataset contain all possible instances or is it a sample (not necessarily random) of instances from a larger set? If the dataset is a sample, then what is the larger set? Is the sample representative of the larger set (e.g., geographic coverage)? If so, please describe how this representativeness was validated/verified. If it is not representative of the larger set, please describe why not (e.g., to cover a more diverse range of instances, because instances were withheld or unavailable). The subset of our dataset that comes from multilingual Common Crawl datasets (MC4, Oscar), are filtered by language to only include Swedish, Norwegian, Danish, and Icelandic. From The Pile, we included only the parts that typically are of highest textual quality or complemented the rest of our dataset with sources we otherwise lacked (e.g. books). The remainder of the dataset was collected from the above sources.
|
| 221 |
+
- What data does each instance consist of? “Raw” data (e.g., unprocessed text or images) or features? In either case, please provide a description. Each instance consists of raw text data.
|
| 222 |
+
- Is there a label or target associated with each instance? If so, please provide a description. No.
|
| 223 |
+
- Is any information missing from individual instances? If so, please provide a description, explaining why this information is missing (e.g., because it was unavailable). This does not include intentionally removed information, but might include, e.g., redacted text. No.
|
| 224 |
+
- Are relationships between individual instances made explicit (e.g., users’ movie ratings, social network links)? If so, please describe how these relationships are made explicit. There are no explicit relationships between individual instances.
|
| 225 |
+
- Are there recommended data splits (e.g., training, development/validation, testing)? If so, please provide a description of these splits, explaining the rationale behind them. There are no explicit splits recommended for this dataset. When pre-training the model, a random split for train, dev, test is set to 99.99%, 0.08%, 0.02% respectively, and is sampled proportionally to each subset’s weight and size. The weight of each subset was manually decided beforehand. These decisions were made considering the data’s value, source, and language, to form a representative and balanced pre-training corpus.
|
| 226 |
+
- Are there any errors, sources of noise, or redundancies in the dataset? If so, please provide a description. The dataset is a collection of many sources, some of which naturally contain some overlap. Although we have performed deduplication, some overlap may still remain. Furthermore, there may be some noise remaining from artifacts originating in Common Crawl datasets, that have been missed by our data filtering process. Except for these, we are not aware of any errors, sources of noise, or redundancies.
|
| 227 |
+
- Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)? The dataset is self-contained.
|
| 228 |
+
- Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety? If so, please describe why. The dataset contains subsets of public Common Crawl, Reddit, Familjeliv and Flashback. These could contain sentences that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety.
|
| 229 |
+
- Does the dataset relate to people? If not, you may skip the remaining questions in this section. Some documents of this data relate to people, such as news articles, Wikipedia descriptions, etc.
|
| 230 |
+
- Does the dataset identify any subpopulations (e.g., by age, gender)? If so, please describe how these subpopulations are identified and provide a description of their respective distributions within the dataset. No, the dataset does not explicitly include subpopulation identification.
|
| 231 |
+
- Any other comments? No.
|
| 232 |
+
|
| 233 |
+
# Collection Process
|
| 234 |
+
- How was the data associated with each instance acquired? Was the data directly observable (e.g., raw text, movie ratings), reported by subjects (e.g., survey responses), or indirectly inferred/derived from other data (e.g., part-of-speech tags, model-based guesses for age or language)? If data was reported by subjects or indirectly inferred/derived from other data, was the data validated/verified? If so, please describe how. N/A. The dataset is a union of publicly available datasets and sources.
|
| 235 |
+
- What mechanisms or procedures were used to collect the data (e.g., hardware apparatus or sensor, manual human curation, software program, software API)? How were these mechanisms or procedures validated? The data was downloaded from the internet.
|
| 236 |
+
- If the dataset is a sample from a larger set, what was the sampling strategy (e.g., deterministic, probabilistic with specific sampling probabilities)? Please see previous answers for how parts of the dataset were selected.
|
| 237 |
+
- Who was involved in the data collection process (e.g., students, crowdworkers, contractors) and how were they compensated (e.g., how much were crowdworkers paid)? This data is mined, filtered and sampled by machines.
|
| 238 |
+
- Over what timeframe was the data collected? Does this timeframe match the creation timeframe of the data associated with the instances (e.g., recent crawl of old news articles)? If not, please describe the timeframe in which the data associated with the instances was created. The dataset was collected during the period June 2021 to June 2022. The creation of the collected sources varies, with e.g. Common Crawl data that have been continuously collected over 12 years.
|
| 239 |
+
- Does the dataset relate to people? If not, you may skip the remainder of the questions in this section. Yes. The texts have been produced by people. Any personal information potentially present in publicly available data sources and thus in the created dataset is of no interest to the collection and use of the dataset.
|
| 240 |
+
- Has an analysis of the potential impact of the dataset and its use on data subjects (e.g., a data protection impact analysis) been conducted? If so, please provide a description of this analysis, including the outcomes, as well as a link or other access point to any supporting documentation. Yes.
|
| 241 |
+
- Any other comments? No.
|
| 242 |
+
- Preprocessing/cleaning/labeling
|
| 243 |
+
- Was any preprocessing/cleaning/labeling of the data done (e.g., discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, removal of instances, processing of missing values)? If so, please provide a description. If not, you may skip the remainder of the questions in this section. The dataset was filtered and re-formatted on a document-level using standard procedures, inspired by the work in The BigScience ROOTS Corpus (H. Laurençon et al., 2022) and Gopher (J. W. Rae et al., 2022). This was done with the goal of achieving a consistent text format throughout the dataset, and to remove documents that did not meet our textual quality requirements (e.g. repetitiveness). Furthermore, the dataset was deduplicated to remedy the overlap between collected subsets using the MinHash algorithm, similar to the method used in GPT-3 and The Pile, and described in greater detail in “Deduplicating Training Data Makes Language Models Better” (K. Lee et al., 2021).
|
| 244 |
+
|
| 245 |
+
**Instruction data**: The processing outlined above was not applied to the instruction data.
|
| 246 |
+
Instruction data was turned into chat-turn format and formatted accordingly with an end-of-turn token, as well as unrolled into raw textual form.
|
| 247 |
+
The Open Assistant data was also automatically translated using GPT-SW3 into Swedish, Danish, Norwegian, and Icelandic.
|
| 248 |
+
- Was the “raw” data saved in addition to the preprocessed/cleaned/labeled data (e.g., to support unanticipated future uses)? If so, please provide a link or other access point to the “raw” data. The “raw” component datasets are publicly available in their respective locations.
|
| 249 |
+
- Any other comments? No.
|
| 250 |
+
|
| 251 |
+
# Uses
|
| 252 |
+
- Has the dataset been used for any tasks already? If so, please provide a description. The dataset was used to pre-train the GPT-SW3 models.
|
| 253 |
+
- Is there a repository that links to any or all papers or systems that use the dataset? If so, please provide a link or other access point. N/A.
|
| 254 |
+
- What (other) tasks could the dataset be used for? The data can be used to pre-train language models, which are foundations for many current and future language tasks.
|
| 255 |
+
- Is there anything about the composition of the dataset or the way it was collected and preprocessed/cleaned/labeled that might impact future uses? For example, is there anything that a future user might need to know to avoid uses that could result in unfair treatment of individuals or groups (e.g., stereotyping, quality of service issues) or other undesirable harms (e.g., financial harms, legal risks) If so, please provide a description. Is there anything a future user could do to mitigate these undesirable harms? The dataset is probably quite representative of Swedish internet discourse in general, and of the Swedish public sector, but we know that this data does not necessarily reflect the entire Swedish population.
|
| 256 |
+
- Are there tasks for which the dataset should not be used? If so, please provide a description. None that we are currently aware of.
|
| 257 |
+
- Any other comments? No.
|
| 258 |
+
|
| 259 |
+
# Distribution
|
| 260 |
+
- Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on behalf of which the dataset was created? If so, please provide a description. No.
|
| 261 |
+
- How will the dataset distributed (e.g., tarball on website, API, GitHub)? Does the dataset have a digital object identifier (DOI)? N/A.
|
| 262 |
+
- When will the dataset be distributed? N/A.
|
| 263 |
+
- Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under applicable terms of use (ToU)? If so, please describe this license and/or ToU, and provide a link or other access point to, or otherwise reproduce, any relevant licensing terms or ToU, as well as any fees associated with these restrictions. N/A.
|
| 264 |
+
- Do any export controls or other regulatory restrictions apply to the dataset or to individual instances? If so, please describe these restrictions, and provide a link or other access point to, or otherwise reproduce, any supporting documentation. N/A.
|
| 265 |
+
- Any other comments? No.
|
| 266 |
+
|
| 267 |
+
# Maintenance
|
| 268 |
+
- Who is supporting/hosting/maintaining the dataset? AI Sweden at Lindholmen Science Park AB.
|
| 269 |
+
- How can the owner/curator/manager of the dataset be contacted (e.g., email address)? nlu@ai.se
|
| 270 |
+
- Is there an erratum? If so, please provide a link or other access point. N/A.
|
| 271 |
+
- Will the dataset be updated (e.g., to correct labeling errors, add new instances, delete instances)? If so, please describe how often, by whom, and how updates will be communicated to users (e.g., mailing list, GitHub)? Currently, there are no plans for updating the dataset.
|
| 272 |
+
- If the dataset relates to people, are there applicable limits on the retention of the data associated with the instances (e.g., were individuals in question told that their data would be retained for a fixed period of time and then deleted)? If so, please describe these limits and explain how they will be enforced. Read the privacy policy for the NLU initiative at AI Sweden [here](https://www.ai.se/en/privacy-policy-nlu).
|
| 273 |
+
- Will older versions of the dataset continue to be supported/hosted/maintained? If so, please describe how. If not, please describe how its obsolescence will be communicated to users. N/A.
|
| 274 |
+
- If others want to extend/augment/build on/contribute to the dataset, is there a mechanism for them to do so? If so, please provide a description. Will these contributions be validated/ verified? If so, please describe how. If not, why not? Is there a process for communicating/ distributing these contributions to other users? If so, please provide a description. Not at this time.
|
| 275 |
+
- Any other comments? No.
|
| 276 |
+
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
| 277 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_AI-Sweden-Models__gpt-sw3-6.7b-v2-instruct)
|
| 278 |
+
|
| 279 |
+
| Metric | Value |
|
| 280 |
+
|-----------------------|---------------------------|
|
| 281 |
+
| Avg. | 39.57 |
|
| 282 |
+
| ARC (25-shot) | 40.78 |
|
| 283 |
+
| HellaSwag (10-shot) | 67.77 |
|
| 284 |
+
| MMLU (5-shot) | 31.57 |
|
| 285 |
+
| TruthfulQA (0-shot) | 40.32 |
|
| 286 |
+
| Winogrande (5-shot) | 63.54 |
|
| 287 |
+
| GSM8K (5-shot) | 6.37 |
|
| 288 |
+
| DROP (3-shot) | 26.67 |
|
config.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_name_or_path": "/home/ariel/gpt_sw3/instruct_new/7b/hf/",
|
| 3 |
+
"activation_function": "gelu",
|
| 4 |
+
"apply_query_key_layer_scaling": true,
|
| 5 |
+
"architectures": [
|
| 6 |
+
"GPT2LMHeadModel"
|
| 7 |
+
],
|
| 8 |
+
"attn_pdrop": 0.1,
|
| 9 |
+
"bos_token_id": 1,
|
| 10 |
+
"embd_pdrop": 0.1,
|
| 11 |
+
"eos_token_id": 1,
|
| 12 |
+
"initializer_range": 0.01,
|
| 13 |
+
"layer_norm_epsilon": 1e-05,
|
| 14 |
+
"model_type": "gpt2",
|
| 15 |
+
"n_embd": 4096,
|
| 16 |
+
"n_head": 32,
|
| 17 |
+
"n_inner": 16384,
|
| 18 |
+
"n_layer": 32,
|
| 19 |
+
"n_positions": 2048,
|
| 20 |
+
"normalize_attention_scores": true,
|
| 21 |
+
"pad_token_id": 0,
|
| 22 |
+
"reorder_and_upcast_attn": false,
|
| 23 |
+
"resid_pdrop": 0.1,
|
| 24 |
+
"scale_attn_by_inverse_layer_idx": false,
|
| 25 |
+
"scale_attn_weights": true,
|
| 26 |
+
"summary_activation": null,
|
| 27 |
+
"summary_first_dropout": 0.1,
|
| 28 |
+
"summary_proj_to_labels": true,
|
| 29 |
+
"summary_type": "cls_index",
|
| 30 |
+
"summary_use_proj": true,
|
| 31 |
+
"tokenizer_class": "GPTSw3Tokenizer",
|
| 32 |
+
"torch_dtype": "float32",
|
| 33 |
+
"transformers_version": "4.22.1",
|
| 34 |
+
"use_cache": true,
|
| 35 |
+
"vocab_size": 64000,
|
| 36 |
+
"max_length": 2048
|
| 37 |
+
}
|
model-00001-of-00003.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:65b0eec0c89b1cf4f7b92279e27469bce17430286a58267ec37b64e1942c30ce
|
| 3 |
+
size 9993224624
|
model-00002-of-00003.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:656723c8ac9c21da428063733e0323d5d95238178541fc45e37a3845e30368f4
|
| 3 |
+
size 9985116208
|
model-00003-of-00003.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:816d394b0993a2bb22c812bccdcb6d8a473c03fb8e2d9581292ab4112568078f
|
| 3 |
+
size 8063283552
|
model.safetensors.index.json
ADDED
|
@@ -0,0 +1,460 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"metadata": {
|
| 3 |
+
"total_size": 28041576576
|
| 4 |
+
},
|
| 5 |
+
"weight_map": {
|
| 6 |
+
"lm_head.weight": "model-00003-of-00003.safetensors",
|
| 7 |
+
"transformer.h.0.attn.bias": "model-00001-of-00003.safetensors",
|
| 8 |
+
"transformer.h.0.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 9 |
+
"transformer.h.0.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 10 |
+
"transformer.h.0.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 11 |
+
"transformer.h.0.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 12 |
+
"transformer.h.0.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 13 |
+
"transformer.h.0.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 14 |
+
"transformer.h.0.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 15 |
+
"transformer.h.0.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 16 |
+
"transformer.h.0.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 17 |
+
"transformer.h.0.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 18 |
+
"transformer.h.0.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 19 |
+
"transformer.h.0.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 20 |
+
"transformer.h.0.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 21 |
+
"transformer.h.1.attn.bias": "model-00001-of-00003.safetensors",
|
| 22 |
+
"transformer.h.1.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 23 |
+
"transformer.h.1.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 24 |
+
"transformer.h.1.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 25 |
+
"transformer.h.1.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 26 |
+
"transformer.h.1.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 27 |
+
"transformer.h.1.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 28 |
+
"transformer.h.1.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 29 |
+
"transformer.h.1.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 30 |
+
"transformer.h.1.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 31 |
+
"transformer.h.1.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 32 |
+
"transformer.h.1.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 33 |
+
"transformer.h.1.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 34 |
+
"transformer.h.1.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 35 |
+
"transformer.h.10.attn.bias": "model-00001-of-00003.safetensors",
|
| 36 |
+
"transformer.h.10.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 37 |
+
"transformer.h.10.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 38 |
+
"transformer.h.10.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 39 |
+
"transformer.h.10.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 40 |
+
"transformer.h.10.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 41 |
+
"transformer.h.10.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 42 |
+
"transformer.h.10.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 43 |
+
"transformer.h.10.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 44 |
+
"transformer.h.10.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 45 |
+
"transformer.h.10.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 46 |
+
"transformer.h.10.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 47 |
+
"transformer.h.10.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 48 |
+
"transformer.h.10.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 49 |
+
"transformer.h.11.attn.bias": "model-00001-of-00003.safetensors",
|
| 50 |
+
"transformer.h.11.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 51 |
+
"transformer.h.11.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 52 |
+
"transformer.h.11.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 53 |
+
"transformer.h.11.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 54 |
+
"transformer.h.11.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 55 |
+
"transformer.h.11.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 56 |
+
"transformer.h.11.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 57 |
+
"transformer.h.11.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 58 |
+
"transformer.h.11.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 59 |
+
"transformer.h.11.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 60 |
+
"transformer.h.11.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 61 |
+
"transformer.h.11.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 62 |
+
"transformer.h.11.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 63 |
+
"transformer.h.12.attn.bias": "model-00002-of-00003.safetensors",
|
| 64 |
+
"transformer.h.12.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 65 |
+
"transformer.h.12.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 66 |
+
"transformer.h.12.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 67 |
+
"transformer.h.12.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 68 |
+
"transformer.h.12.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 69 |
+
"transformer.h.12.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 70 |
+
"transformer.h.12.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 71 |
+
"transformer.h.12.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 72 |
+
"transformer.h.12.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 73 |
+
"transformer.h.12.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 74 |
+
"transformer.h.12.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 75 |
+
"transformer.h.12.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 76 |
+
"transformer.h.12.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 77 |
+
"transformer.h.13.attn.bias": "model-00002-of-00003.safetensors",
|
| 78 |
+
"transformer.h.13.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 79 |
+
"transformer.h.13.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 80 |
+
"transformer.h.13.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 81 |
+
"transformer.h.13.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 82 |
+
"transformer.h.13.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 83 |
+
"transformer.h.13.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 84 |
+
"transformer.h.13.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 85 |
+
"transformer.h.13.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 86 |
+
"transformer.h.13.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 87 |
+
"transformer.h.13.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 88 |
+
"transformer.h.13.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 89 |
+
"transformer.h.13.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 90 |
+
"transformer.h.13.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 91 |
+
"transformer.h.14.attn.bias": "model-00002-of-00003.safetensors",
|
| 92 |
+
"transformer.h.14.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 93 |
+
"transformer.h.14.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 94 |
+
"transformer.h.14.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 95 |
+
"transformer.h.14.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 96 |
+
"transformer.h.14.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 97 |
+
"transformer.h.14.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 98 |
+
"transformer.h.14.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 99 |
+
"transformer.h.14.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 100 |
+
"transformer.h.14.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 101 |
+
"transformer.h.14.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 102 |
+
"transformer.h.14.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 103 |
+
"transformer.h.14.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 104 |
+
"transformer.h.14.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 105 |
+
"transformer.h.15.attn.bias": "model-00002-of-00003.safetensors",
|
| 106 |
+
"transformer.h.15.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 107 |
+
"transformer.h.15.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 108 |
+
"transformer.h.15.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 109 |
+
"transformer.h.15.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 110 |
+
"transformer.h.15.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 111 |
+
"transformer.h.15.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 112 |
+
"transformer.h.15.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 113 |
+
"transformer.h.15.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 114 |
+
"transformer.h.15.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 115 |
+
"transformer.h.15.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 116 |
+
"transformer.h.15.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 117 |
+
"transformer.h.15.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 118 |
+
"transformer.h.15.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 119 |
+
"transformer.h.16.attn.bias": "model-00002-of-00003.safetensors",
|
| 120 |
+
"transformer.h.16.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 121 |
+
"transformer.h.16.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 122 |
+
"transformer.h.16.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 123 |
+
"transformer.h.16.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 124 |
+
"transformer.h.16.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 125 |
+
"transformer.h.16.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 126 |
+
"transformer.h.16.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 127 |
+
"transformer.h.16.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 128 |
+
"transformer.h.16.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 129 |
+
"transformer.h.16.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 130 |
+
"transformer.h.16.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 131 |
+
"transformer.h.16.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 132 |
+
"transformer.h.16.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 133 |
+
"transformer.h.17.attn.bias": "model-00002-of-00003.safetensors",
|
| 134 |
+
"transformer.h.17.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 135 |
+
"transformer.h.17.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 136 |
+
"transformer.h.17.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 137 |
+
"transformer.h.17.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 138 |
+
"transformer.h.17.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 139 |
+
"transformer.h.17.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 140 |
+
"transformer.h.17.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 141 |
+
"transformer.h.17.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 142 |
+
"transformer.h.17.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 143 |
+
"transformer.h.17.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 144 |
+
"transformer.h.17.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 145 |
+
"transformer.h.17.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 146 |
+
"transformer.h.17.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 147 |
+
"transformer.h.18.attn.bias": "model-00002-of-00003.safetensors",
|
| 148 |
+
"transformer.h.18.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 149 |
+
"transformer.h.18.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 150 |
+
"transformer.h.18.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 151 |
+
"transformer.h.18.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 152 |
+
"transformer.h.18.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 153 |
+
"transformer.h.18.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 154 |
+
"transformer.h.18.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 155 |
+
"transformer.h.18.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 156 |
+
"transformer.h.18.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 157 |
+
"transformer.h.18.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 158 |
+
"transformer.h.18.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 159 |
+
"transformer.h.18.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 160 |
+
"transformer.h.18.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 161 |
+
"transformer.h.19.attn.bias": "model-00002-of-00003.safetensors",
|
| 162 |
+
"transformer.h.19.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 163 |
+
"transformer.h.19.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 164 |
+
"transformer.h.19.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 165 |
+
"transformer.h.19.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 166 |
+
"transformer.h.19.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 167 |
+
"transformer.h.19.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 168 |
+
"transformer.h.19.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 169 |
+
"transformer.h.19.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 170 |
+
"transformer.h.19.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 171 |
+
"transformer.h.19.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 172 |
+
"transformer.h.19.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 173 |
+
"transformer.h.19.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 174 |
+
"transformer.h.19.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 175 |
+
"transformer.h.2.attn.bias": "model-00001-of-00003.safetensors",
|
| 176 |
+
"transformer.h.2.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 177 |
+
"transformer.h.2.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 178 |
+
"transformer.h.2.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 179 |
+
"transformer.h.2.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 180 |
+
"transformer.h.2.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 181 |
+
"transformer.h.2.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 182 |
+
"transformer.h.2.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 183 |
+
"transformer.h.2.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 184 |
+
"transformer.h.2.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 185 |
+
"transformer.h.2.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 186 |
+
"transformer.h.2.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 187 |
+
"transformer.h.2.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 188 |
+
"transformer.h.2.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 189 |
+
"transformer.h.20.attn.bias": "model-00002-of-00003.safetensors",
|
| 190 |
+
"transformer.h.20.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 191 |
+
"transformer.h.20.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 192 |
+
"transformer.h.20.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 193 |
+
"transformer.h.20.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 194 |
+
"transformer.h.20.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 195 |
+
"transformer.h.20.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 196 |
+
"transformer.h.20.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 197 |
+
"transformer.h.20.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 198 |
+
"transformer.h.20.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 199 |
+
"transformer.h.20.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 200 |
+
"transformer.h.20.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 201 |
+
"transformer.h.20.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 202 |
+
"transformer.h.20.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 203 |
+
"transformer.h.21.attn.bias": "model-00002-of-00003.safetensors",
|
| 204 |
+
"transformer.h.21.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 205 |
+
"transformer.h.21.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 206 |
+
"transformer.h.21.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 207 |
+
"transformer.h.21.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 208 |
+
"transformer.h.21.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 209 |
+
"transformer.h.21.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 210 |
+
"transformer.h.21.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 211 |
+
"transformer.h.21.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 212 |
+
"transformer.h.21.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 213 |
+
"transformer.h.21.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 214 |
+
"transformer.h.21.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 215 |
+
"transformer.h.21.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 216 |
+
"transformer.h.21.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 217 |
+
"transformer.h.22.attn.bias": "model-00002-of-00003.safetensors",
|
| 218 |
+
"transformer.h.22.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 219 |
+
"transformer.h.22.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 220 |
+
"transformer.h.22.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 221 |
+
"transformer.h.22.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 222 |
+
"transformer.h.22.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 223 |
+
"transformer.h.22.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 224 |
+
"transformer.h.22.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 225 |
+
"transformer.h.22.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 226 |
+
"transformer.h.22.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 227 |
+
"transformer.h.22.mlp.c_fc.bias": "model-00002-of-00003.safetensors",
|
| 228 |
+
"transformer.h.22.mlp.c_fc.weight": "model-00002-of-00003.safetensors",
|
| 229 |
+
"transformer.h.22.mlp.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 230 |
+
"transformer.h.22.mlp.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 231 |
+
"transformer.h.23.attn.bias": "model-00002-of-00003.safetensors",
|
| 232 |
+
"transformer.h.23.attn.c_attn.bias": "model-00002-of-00003.safetensors",
|
| 233 |
+
"transformer.h.23.attn.c_attn.weight": "model-00002-of-00003.safetensors",
|
| 234 |
+
"transformer.h.23.attn.c_proj.bias": "model-00002-of-00003.safetensors",
|
| 235 |
+
"transformer.h.23.attn.c_proj.weight": "model-00002-of-00003.safetensors",
|
| 236 |
+
"transformer.h.23.attn.masked_bias": "model-00002-of-00003.safetensors",
|
| 237 |
+
"transformer.h.23.ln_1.bias": "model-00002-of-00003.safetensors",
|
| 238 |
+
"transformer.h.23.ln_1.weight": "model-00002-of-00003.safetensors",
|
| 239 |
+
"transformer.h.23.ln_2.bias": "model-00002-of-00003.safetensors",
|
| 240 |
+
"transformer.h.23.ln_2.weight": "model-00002-of-00003.safetensors",
|
| 241 |
+
"transformer.h.23.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 242 |
+
"transformer.h.23.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 243 |
+
"transformer.h.23.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 244 |
+
"transformer.h.23.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 245 |
+
"transformer.h.24.attn.bias": "model-00003-of-00003.safetensors",
|
| 246 |
+
"transformer.h.24.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 247 |
+
"transformer.h.24.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 248 |
+
"transformer.h.24.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 249 |
+
"transformer.h.24.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 250 |
+
"transformer.h.24.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 251 |
+
"transformer.h.24.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 252 |
+
"transformer.h.24.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 253 |
+
"transformer.h.24.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 254 |
+
"transformer.h.24.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 255 |
+
"transformer.h.24.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 256 |
+
"transformer.h.24.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 257 |
+
"transformer.h.24.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 258 |
+
"transformer.h.24.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 259 |
+
"transformer.h.25.attn.bias": "model-00003-of-00003.safetensors",
|
| 260 |
+
"transformer.h.25.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 261 |
+
"transformer.h.25.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 262 |
+
"transformer.h.25.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 263 |
+
"transformer.h.25.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 264 |
+
"transformer.h.25.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 265 |
+
"transformer.h.25.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 266 |
+
"transformer.h.25.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 267 |
+
"transformer.h.25.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 268 |
+
"transformer.h.25.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 269 |
+
"transformer.h.25.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 270 |
+
"transformer.h.25.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 271 |
+
"transformer.h.25.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 272 |
+
"transformer.h.25.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 273 |
+
"transformer.h.26.attn.bias": "model-00003-of-00003.safetensors",
|
| 274 |
+
"transformer.h.26.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 275 |
+
"transformer.h.26.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 276 |
+
"transformer.h.26.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 277 |
+
"transformer.h.26.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 278 |
+
"transformer.h.26.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 279 |
+
"transformer.h.26.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 280 |
+
"transformer.h.26.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 281 |
+
"transformer.h.26.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 282 |
+
"transformer.h.26.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 283 |
+
"transformer.h.26.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 284 |
+
"transformer.h.26.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 285 |
+
"transformer.h.26.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 286 |
+
"transformer.h.26.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 287 |
+
"transformer.h.27.attn.bias": "model-00003-of-00003.safetensors",
|
| 288 |
+
"transformer.h.27.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 289 |
+
"transformer.h.27.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 290 |
+
"transformer.h.27.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 291 |
+
"transformer.h.27.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 292 |
+
"transformer.h.27.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 293 |
+
"transformer.h.27.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 294 |
+
"transformer.h.27.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 295 |
+
"transformer.h.27.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 296 |
+
"transformer.h.27.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 297 |
+
"transformer.h.27.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 298 |
+
"transformer.h.27.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 299 |
+
"transformer.h.27.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 300 |
+
"transformer.h.27.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 301 |
+
"transformer.h.28.attn.bias": "model-00003-of-00003.safetensors",
|
| 302 |
+
"transformer.h.28.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 303 |
+
"transformer.h.28.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 304 |
+
"transformer.h.28.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 305 |
+
"transformer.h.28.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 306 |
+
"transformer.h.28.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 307 |
+
"transformer.h.28.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 308 |
+
"transformer.h.28.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 309 |
+
"transformer.h.28.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 310 |
+
"transformer.h.28.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 311 |
+
"transformer.h.28.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 312 |
+
"transformer.h.28.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 313 |
+
"transformer.h.28.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 314 |
+
"transformer.h.28.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 315 |
+
"transformer.h.29.attn.bias": "model-00003-of-00003.safetensors",
|
| 316 |
+
"transformer.h.29.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 317 |
+
"transformer.h.29.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 318 |
+
"transformer.h.29.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 319 |
+
"transformer.h.29.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 320 |
+
"transformer.h.29.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 321 |
+
"transformer.h.29.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 322 |
+
"transformer.h.29.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 323 |
+
"transformer.h.29.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 324 |
+
"transformer.h.29.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 325 |
+
"transformer.h.29.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 326 |
+
"transformer.h.29.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 327 |
+
"transformer.h.29.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 328 |
+
"transformer.h.29.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 329 |
+
"transformer.h.3.attn.bias": "model-00001-of-00003.safetensors",
|
| 330 |
+
"transformer.h.3.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 331 |
+
"transformer.h.3.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 332 |
+
"transformer.h.3.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 333 |
+
"transformer.h.3.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 334 |
+
"transformer.h.3.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 335 |
+
"transformer.h.3.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 336 |
+
"transformer.h.3.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 337 |
+
"transformer.h.3.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 338 |
+
"transformer.h.3.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 339 |
+
"transformer.h.3.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 340 |
+
"transformer.h.3.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 341 |
+
"transformer.h.3.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 342 |
+
"transformer.h.3.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 343 |
+
"transformer.h.30.attn.bias": "model-00003-of-00003.safetensors",
|
| 344 |
+
"transformer.h.30.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 345 |
+
"transformer.h.30.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 346 |
+
"transformer.h.30.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 347 |
+
"transformer.h.30.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 348 |
+
"transformer.h.30.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 349 |
+
"transformer.h.30.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 350 |
+
"transformer.h.30.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 351 |
+
"transformer.h.30.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 352 |
+
"transformer.h.30.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 353 |
+
"transformer.h.30.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 354 |
+
"transformer.h.30.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 355 |
+
"transformer.h.30.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 356 |
+
"transformer.h.30.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 357 |
+
"transformer.h.31.attn.bias": "model-00003-of-00003.safetensors",
|
| 358 |
+
"transformer.h.31.attn.c_attn.bias": "model-00003-of-00003.safetensors",
|
| 359 |
+
"transformer.h.31.attn.c_attn.weight": "model-00003-of-00003.safetensors",
|
| 360 |
+
"transformer.h.31.attn.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 361 |
+
"transformer.h.31.attn.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 362 |
+
"transformer.h.31.attn.masked_bias": "model-00003-of-00003.safetensors",
|
| 363 |
+
"transformer.h.31.ln_1.bias": "model-00003-of-00003.safetensors",
|
| 364 |
+
"transformer.h.31.ln_1.weight": "model-00003-of-00003.safetensors",
|
| 365 |
+
"transformer.h.31.ln_2.bias": "model-00003-of-00003.safetensors",
|
| 366 |
+
"transformer.h.31.ln_2.weight": "model-00003-of-00003.safetensors",
|
| 367 |
+
"transformer.h.31.mlp.c_fc.bias": "model-00003-of-00003.safetensors",
|
| 368 |
+
"transformer.h.31.mlp.c_fc.weight": "model-00003-of-00003.safetensors",
|
| 369 |
+
"transformer.h.31.mlp.c_proj.bias": "model-00003-of-00003.safetensors",
|
| 370 |
+
"transformer.h.31.mlp.c_proj.weight": "model-00003-of-00003.safetensors",
|
| 371 |
+
"transformer.h.4.attn.bias": "model-00001-of-00003.safetensors",
|
| 372 |
+
"transformer.h.4.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 373 |
+
"transformer.h.4.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 374 |
+
"transformer.h.4.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 375 |
+
"transformer.h.4.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 376 |
+
"transformer.h.4.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 377 |
+
"transformer.h.4.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 378 |
+
"transformer.h.4.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 379 |
+
"transformer.h.4.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 380 |
+
"transformer.h.4.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 381 |
+
"transformer.h.4.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 382 |
+
"transformer.h.4.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 383 |
+
"transformer.h.4.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 384 |
+
"transformer.h.4.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 385 |
+
"transformer.h.5.attn.bias": "model-00001-of-00003.safetensors",
|
| 386 |
+
"transformer.h.5.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 387 |
+
"transformer.h.5.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 388 |
+
"transformer.h.5.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 389 |
+
"transformer.h.5.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 390 |
+
"transformer.h.5.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 391 |
+
"transformer.h.5.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 392 |
+
"transformer.h.5.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 393 |
+
"transformer.h.5.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 394 |
+
"transformer.h.5.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 395 |
+
"transformer.h.5.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 396 |
+
"transformer.h.5.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 397 |
+
"transformer.h.5.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 398 |
+
"transformer.h.5.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 399 |
+
"transformer.h.6.attn.bias": "model-00001-of-00003.safetensors",
|
| 400 |
+
"transformer.h.6.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 401 |
+
"transformer.h.6.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 402 |
+
"transformer.h.6.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 403 |
+
"transformer.h.6.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 404 |
+
"transformer.h.6.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 405 |
+
"transformer.h.6.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 406 |
+
"transformer.h.6.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 407 |
+
"transformer.h.6.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 408 |
+
"transformer.h.6.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 409 |
+
"transformer.h.6.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 410 |
+
"transformer.h.6.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 411 |
+
"transformer.h.6.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 412 |
+
"transformer.h.6.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 413 |
+
"transformer.h.7.attn.bias": "model-00001-of-00003.safetensors",
|
| 414 |
+
"transformer.h.7.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 415 |
+
"transformer.h.7.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 416 |
+
"transformer.h.7.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 417 |
+
"transformer.h.7.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 418 |
+
"transformer.h.7.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 419 |
+
"transformer.h.7.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 420 |
+
"transformer.h.7.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 421 |
+
"transformer.h.7.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 422 |
+
"transformer.h.7.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 423 |
+
"transformer.h.7.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 424 |
+
"transformer.h.7.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 425 |
+
"transformer.h.7.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 426 |
+
"transformer.h.7.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 427 |
+
"transformer.h.8.attn.bias": "model-00001-of-00003.safetensors",
|
| 428 |
+
"transformer.h.8.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 429 |
+
"transformer.h.8.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 430 |
+
"transformer.h.8.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 431 |
+
"transformer.h.8.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 432 |
+
"transformer.h.8.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 433 |
+
"transformer.h.8.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 434 |
+
"transformer.h.8.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 435 |
+
"transformer.h.8.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 436 |
+
"transformer.h.8.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 437 |
+
"transformer.h.8.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 438 |
+
"transformer.h.8.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 439 |
+
"transformer.h.8.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 440 |
+
"transformer.h.8.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 441 |
+
"transformer.h.9.attn.bias": "model-00001-of-00003.safetensors",
|
| 442 |
+
"transformer.h.9.attn.c_attn.bias": "model-00001-of-00003.safetensors",
|
| 443 |
+
"transformer.h.9.attn.c_attn.weight": "model-00001-of-00003.safetensors",
|
| 444 |
+
"transformer.h.9.attn.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 445 |
+
"transformer.h.9.attn.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 446 |
+
"transformer.h.9.attn.masked_bias": "model-00001-of-00003.safetensors",
|
| 447 |
+
"transformer.h.9.ln_1.bias": "model-00001-of-00003.safetensors",
|
| 448 |
+
"transformer.h.9.ln_1.weight": "model-00001-of-00003.safetensors",
|
| 449 |
+
"transformer.h.9.ln_2.bias": "model-00001-of-00003.safetensors",
|
| 450 |
+
"transformer.h.9.ln_2.weight": "model-00001-of-00003.safetensors",
|
| 451 |
+
"transformer.h.9.mlp.c_fc.bias": "model-00001-of-00003.safetensors",
|
| 452 |
+
"transformer.h.9.mlp.c_fc.weight": "model-00001-of-00003.safetensors",
|
| 453 |
+
"transformer.h.9.mlp.c_proj.bias": "model-00001-of-00003.safetensors",
|
| 454 |
+
"transformer.h.9.mlp.c_proj.weight": "model-00001-of-00003.safetensors",
|
| 455 |
+
"transformer.ln_f.bias": "model-00003-of-00003.safetensors",
|
| 456 |
+
"transformer.ln_f.weight": "model-00003-of-00003.safetensors",
|
| 457 |
+
"transformer.wpe.weight": "model-00001-of-00003.safetensors",
|
| 458 |
+
"transformer.wte.weight": "model-00001-of-00003.safetensors"
|
| 459 |
+
}
|
| 460 |
+
}
|
pytorch_model-00001-of-00003.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:91d5d5aeec9ba8275f62f08e7a77b9d67ee09a640b7b1ead73d2ce346642ca1c
|
| 3 |
+
size 9993257653
|
pytorch_model-00002-of-00003.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9befff426a8cc6a12b1efd01b1f8e49b99e684ad2ab6c54282d4fb036fc51bfa
|
| 3 |
+
size 9985152001
|
pytorch_model-00003-of-00003.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82fd45f20ed449cfe8abcd4960a2611f503bf49f05aae156cd6006a7edc39ba8
|
| 3 |
+
size 8063307659
|
pytorch_model.bin.index.json
ADDED
|
@@ -0,0 +1,460 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"metadata": {
|
| 3 |
+
"total_size": 28041576576
|
| 4 |
+
},
|
| 5 |
+
"weight_map": {
|
| 6 |
+
"lm_head.weight": "pytorch_model-00003-of-00003.bin",
|
| 7 |
+
"transformer.h.0.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 8 |
+
"transformer.h.0.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 9 |
+
"transformer.h.0.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 10 |
+
"transformer.h.0.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 11 |
+
"transformer.h.0.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 12 |
+
"transformer.h.0.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 13 |
+
"transformer.h.0.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 14 |
+
"transformer.h.0.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 15 |
+
"transformer.h.0.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 16 |
+
"transformer.h.0.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 17 |
+
"transformer.h.0.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 18 |
+
"transformer.h.0.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 19 |
+
"transformer.h.0.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 20 |
+
"transformer.h.0.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 21 |
+
"transformer.h.1.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 22 |
+
"transformer.h.1.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 23 |
+
"transformer.h.1.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 24 |
+
"transformer.h.1.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 25 |
+
"transformer.h.1.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 26 |
+
"transformer.h.1.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 27 |
+
"transformer.h.1.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 28 |
+
"transformer.h.1.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 29 |
+
"transformer.h.1.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 30 |
+
"transformer.h.1.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 31 |
+
"transformer.h.1.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 32 |
+
"transformer.h.1.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 33 |
+
"transformer.h.1.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 34 |
+
"transformer.h.1.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 35 |
+
"transformer.h.10.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 36 |
+
"transformer.h.10.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 37 |
+
"transformer.h.10.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 38 |
+
"transformer.h.10.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 39 |
+
"transformer.h.10.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 40 |
+
"transformer.h.10.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 41 |
+
"transformer.h.10.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 42 |
+
"transformer.h.10.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 43 |
+
"transformer.h.10.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 44 |
+
"transformer.h.10.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 45 |
+
"transformer.h.10.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 46 |
+
"transformer.h.10.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 47 |
+
"transformer.h.10.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 48 |
+
"transformer.h.10.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 49 |
+
"transformer.h.11.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 50 |
+
"transformer.h.11.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 51 |
+
"transformer.h.11.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 52 |
+
"transformer.h.11.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 53 |
+
"transformer.h.11.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 54 |
+
"transformer.h.11.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 55 |
+
"transformer.h.11.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 56 |
+
"transformer.h.11.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 57 |
+
"transformer.h.11.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 58 |
+
"transformer.h.11.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 59 |
+
"transformer.h.11.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 60 |
+
"transformer.h.11.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 61 |
+
"transformer.h.11.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 62 |
+
"transformer.h.11.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 63 |
+
"transformer.h.12.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 64 |
+
"transformer.h.12.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 65 |
+
"transformer.h.12.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 66 |
+
"transformer.h.12.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 67 |
+
"transformer.h.12.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 68 |
+
"transformer.h.12.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 69 |
+
"transformer.h.12.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 70 |
+
"transformer.h.12.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 71 |
+
"transformer.h.12.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 72 |
+
"transformer.h.12.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 73 |
+
"transformer.h.12.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 74 |
+
"transformer.h.12.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 75 |
+
"transformer.h.12.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 76 |
+
"transformer.h.12.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 77 |
+
"transformer.h.13.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 78 |
+
"transformer.h.13.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 79 |
+
"transformer.h.13.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 80 |
+
"transformer.h.13.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 81 |
+
"transformer.h.13.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 82 |
+
"transformer.h.13.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 83 |
+
"transformer.h.13.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 84 |
+
"transformer.h.13.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 85 |
+
"transformer.h.13.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 86 |
+
"transformer.h.13.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 87 |
+
"transformer.h.13.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 88 |
+
"transformer.h.13.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 89 |
+
"transformer.h.13.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 90 |
+
"transformer.h.13.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 91 |
+
"transformer.h.14.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 92 |
+
"transformer.h.14.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 93 |
+
"transformer.h.14.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 94 |
+
"transformer.h.14.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 95 |
+
"transformer.h.14.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 96 |
+
"transformer.h.14.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 97 |
+
"transformer.h.14.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 98 |
+
"transformer.h.14.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 99 |
+
"transformer.h.14.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 100 |
+
"transformer.h.14.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 101 |
+
"transformer.h.14.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 102 |
+
"transformer.h.14.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 103 |
+
"transformer.h.14.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 104 |
+
"transformer.h.14.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 105 |
+
"transformer.h.15.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 106 |
+
"transformer.h.15.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 107 |
+
"transformer.h.15.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 108 |
+
"transformer.h.15.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 109 |
+
"transformer.h.15.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 110 |
+
"transformer.h.15.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 111 |
+
"transformer.h.15.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 112 |
+
"transformer.h.15.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 113 |
+
"transformer.h.15.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 114 |
+
"transformer.h.15.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 115 |
+
"transformer.h.15.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 116 |
+
"transformer.h.15.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 117 |
+
"transformer.h.15.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 118 |
+
"transformer.h.15.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 119 |
+
"transformer.h.16.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 120 |
+
"transformer.h.16.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 121 |
+
"transformer.h.16.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 122 |
+
"transformer.h.16.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 123 |
+
"transformer.h.16.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 124 |
+
"transformer.h.16.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 125 |
+
"transformer.h.16.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 126 |
+
"transformer.h.16.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 127 |
+
"transformer.h.16.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 128 |
+
"transformer.h.16.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 129 |
+
"transformer.h.16.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 130 |
+
"transformer.h.16.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 131 |
+
"transformer.h.16.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 132 |
+
"transformer.h.16.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 133 |
+
"transformer.h.17.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 134 |
+
"transformer.h.17.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 135 |
+
"transformer.h.17.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 136 |
+
"transformer.h.17.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 137 |
+
"transformer.h.17.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 138 |
+
"transformer.h.17.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 139 |
+
"transformer.h.17.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 140 |
+
"transformer.h.17.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 141 |
+
"transformer.h.17.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 142 |
+
"transformer.h.17.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 143 |
+
"transformer.h.17.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 144 |
+
"transformer.h.17.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 145 |
+
"transformer.h.17.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 146 |
+
"transformer.h.17.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 147 |
+
"transformer.h.18.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 148 |
+
"transformer.h.18.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 149 |
+
"transformer.h.18.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 150 |
+
"transformer.h.18.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 151 |
+
"transformer.h.18.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 152 |
+
"transformer.h.18.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 153 |
+
"transformer.h.18.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 154 |
+
"transformer.h.18.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 155 |
+
"transformer.h.18.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 156 |
+
"transformer.h.18.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 157 |
+
"transformer.h.18.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 158 |
+
"transformer.h.18.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 159 |
+
"transformer.h.18.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 160 |
+
"transformer.h.18.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 161 |
+
"transformer.h.19.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 162 |
+
"transformer.h.19.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 163 |
+
"transformer.h.19.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 164 |
+
"transformer.h.19.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 165 |
+
"transformer.h.19.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 166 |
+
"transformer.h.19.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 167 |
+
"transformer.h.19.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 168 |
+
"transformer.h.19.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 169 |
+
"transformer.h.19.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 170 |
+
"transformer.h.19.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 171 |
+
"transformer.h.19.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 172 |
+
"transformer.h.19.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 173 |
+
"transformer.h.19.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 174 |
+
"transformer.h.19.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 175 |
+
"transformer.h.2.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 176 |
+
"transformer.h.2.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 177 |
+
"transformer.h.2.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 178 |
+
"transformer.h.2.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 179 |
+
"transformer.h.2.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 180 |
+
"transformer.h.2.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 181 |
+
"transformer.h.2.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 182 |
+
"transformer.h.2.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 183 |
+
"transformer.h.2.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 184 |
+
"transformer.h.2.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 185 |
+
"transformer.h.2.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 186 |
+
"transformer.h.2.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 187 |
+
"transformer.h.2.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 188 |
+
"transformer.h.2.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 189 |
+
"transformer.h.20.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 190 |
+
"transformer.h.20.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 191 |
+
"transformer.h.20.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 192 |
+
"transformer.h.20.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 193 |
+
"transformer.h.20.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 194 |
+
"transformer.h.20.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 195 |
+
"transformer.h.20.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 196 |
+
"transformer.h.20.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 197 |
+
"transformer.h.20.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 198 |
+
"transformer.h.20.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 199 |
+
"transformer.h.20.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 200 |
+
"transformer.h.20.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 201 |
+
"transformer.h.20.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 202 |
+
"transformer.h.20.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 203 |
+
"transformer.h.21.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 204 |
+
"transformer.h.21.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 205 |
+
"transformer.h.21.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 206 |
+
"transformer.h.21.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 207 |
+
"transformer.h.21.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 208 |
+
"transformer.h.21.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 209 |
+
"transformer.h.21.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 210 |
+
"transformer.h.21.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 211 |
+
"transformer.h.21.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 212 |
+
"transformer.h.21.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 213 |
+
"transformer.h.21.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 214 |
+
"transformer.h.21.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 215 |
+
"transformer.h.21.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 216 |
+
"transformer.h.21.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 217 |
+
"transformer.h.22.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 218 |
+
"transformer.h.22.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 219 |
+
"transformer.h.22.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 220 |
+
"transformer.h.22.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 221 |
+
"transformer.h.22.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 222 |
+
"transformer.h.22.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 223 |
+
"transformer.h.22.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 224 |
+
"transformer.h.22.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 225 |
+
"transformer.h.22.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 226 |
+
"transformer.h.22.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 227 |
+
"transformer.h.22.mlp.c_fc.bias": "pytorch_model-00002-of-00003.bin",
|
| 228 |
+
"transformer.h.22.mlp.c_fc.weight": "pytorch_model-00002-of-00003.bin",
|
| 229 |
+
"transformer.h.22.mlp.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 230 |
+
"transformer.h.22.mlp.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 231 |
+
"transformer.h.23.attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 232 |
+
"transformer.h.23.attn.c_attn.bias": "pytorch_model-00002-of-00003.bin",
|
| 233 |
+
"transformer.h.23.attn.c_attn.weight": "pytorch_model-00002-of-00003.bin",
|
| 234 |
+
"transformer.h.23.attn.c_proj.bias": "pytorch_model-00002-of-00003.bin",
|
| 235 |
+
"transformer.h.23.attn.c_proj.weight": "pytorch_model-00002-of-00003.bin",
|
| 236 |
+
"transformer.h.23.attn.masked_bias": "pytorch_model-00002-of-00003.bin",
|
| 237 |
+
"transformer.h.23.ln_1.bias": "pytorch_model-00002-of-00003.bin",
|
| 238 |
+
"transformer.h.23.ln_1.weight": "pytorch_model-00002-of-00003.bin",
|
| 239 |
+
"transformer.h.23.ln_2.bias": "pytorch_model-00002-of-00003.bin",
|
| 240 |
+
"transformer.h.23.ln_2.weight": "pytorch_model-00002-of-00003.bin",
|
| 241 |
+
"transformer.h.23.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 242 |
+
"transformer.h.23.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 243 |
+
"transformer.h.23.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 244 |
+
"transformer.h.23.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 245 |
+
"transformer.h.24.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 246 |
+
"transformer.h.24.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 247 |
+
"transformer.h.24.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 248 |
+
"transformer.h.24.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 249 |
+
"transformer.h.24.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 250 |
+
"transformer.h.24.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 251 |
+
"transformer.h.24.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 252 |
+
"transformer.h.24.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 253 |
+
"transformer.h.24.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 254 |
+
"transformer.h.24.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 255 |
+
"transformer.h.24.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 256 |
+
"transformer.h.24.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 257 |
+
"transformer.h.24.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 258 |
+
"transformer.h.24.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 259 |
+
"transformer.h.25.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 260 |
+
"transformer.h.25.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 261 |
+
"transformer.h.25.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 262 |
+
"transformer.h.25.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 263 |
+
"transformer.h.25.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 264 |
+
"transformer.h.25.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 265 |
+
"transformer.h.25.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 266 |
+
"transformer.h.25.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 267 |
+
"transformer.h.25.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 268 |
+
"transformer.h.25.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 269 |
+
"transformer.h.25.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 270 |
+
"transformer.h.25.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 271 |
+
"transformer.h.25.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 272 |
+
"transformer.h.25.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 273 |
+
"transformer.h.26.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 274 |
+
"transformer.h.26.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 275 |
+
"transformer.h.26.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 276 |
+
"transformer.h.26.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 277 |
+
"transformer.h.26.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 278 |
+
"transformer.h.26.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 279 |
+
"transformer.h.26.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 280 |
+
"transformer.h.26.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 281 |
+
"transformer.h.26.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 282 |
+
"transformer.h.26.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 283 |
+
"transformer.h.26.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 284 |
+
"transformer.h.26.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 285 |
+
"transformer.h.26.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 286 |
+
"transformer.h.26.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 287 |
+
"transformer.h.27.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 288 |
+
"transformer.h.27.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 289 |
+
"transformer.h.27.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 290 |
+
"transformer.h.27.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 291 |
+
"transformer.h.27.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 292 |
+
"transformer.h.27.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 293 |
+
"transformer.h.27.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 294 |
+
"transformer.h.27.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 295 |
+
"transformer.h.27.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 296 |
+
"transformer.h.27.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 297 |
+
"transformer.h.27.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 298 |
+
"transformer.h.27.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 299 |
+
"transformer.h.27.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 300 |
+
"transformer.h.27.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 301 |
+
"transformer.h.28.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 302 |
+
"transformer.h.28.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 303 |
+
"transformer.h.28.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 304 |
+
"transformer.h.28.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 305 |
+
"transformer.h.28.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 306 |
+
"transformer.h.28.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 307 |
+
"transformer.h.28.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 308 |
+
"transformer.h.28.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 309 |
+
"transformer.h.28.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 310 |
+
"transformer.h.28.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 311 |
+
"transformer.h.28.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 312 |
+
"transformer.h.28.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 313 |
+
"transformer.h.28.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 314 |
+
"transformer.h.28.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 315 |
+
"transformer.h.29.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 316 |
+
"transformer.h.29.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 317 |
+
"transformer.h.29.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 318 |
+
"transformer.h.29.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 319 |
+
"transformer.h.29.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 320 |
+
"transformer.h.29.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 321 |
+
"transformer.h.29.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 322 |
+
"transformer.h.29.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 323 |
+
"transformer.h.29.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 324 |
+
"transformer.h.29.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 325 |
+
"transformer.h.29.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 326 |
+
"transformer.h.29.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 327 |
+
"transformer.h.29.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 328 |
+
"transformer.h.29.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 329 |
+
"transformer.h.3.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 330 |
+
"transformer.h.3.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 331 |
+
"transformer.h.3.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 332 |
+
"transformer.h.3.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 333 |
+
"transformer.h.3.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 334 |
+
"transformer.h.3.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 335 |
+
"transformer.h.3.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 336 |
+
"transformer.h.3.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 337 |
+
"transformer.h.3.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 338 |
+
"transformer.h.3.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 339 |
+
"transformer.h.3.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 340 |
+
"transformer.h.3.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 341 |
+
"transformer.h.3.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 342 |
+
"transformer.h.3.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 343 |
+
"transformer.h.30.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 344 |
+
"transformer.h.30.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 345 |
+
"transformer.h.30.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 346 |
+
"transformer.h.30.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 347 |
+
"transformer.h.30.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 348 |
+
"transformer.h.30.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 349 |
+
"transformer.h.30.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 350 |
+
"transformer.h.30.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 351 |
+
"transformer.h.30.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 352 |
+
"transformer.h.30.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 353 |
+
"transformer.h.30.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 354 |
+
"transformer.h.30.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 355 |
+
"transformer.h.30.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 356 |
+
"transformer.h.30.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 357 |
+
"transformer.h.31.attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 358 |
+
"transformer.h.31.attn.c_attn.bias": "pytorch_model-00003-of-00003.bin",
|
| 359 |
+
"transformer.h.31.attn.c_attn.weight": "pytorch_model-00003-of-00003.bin",
|
| 360 |
+
"transformer.h.31.attn.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 361 |
+
"transformer.h.31.attn.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 362 |
+
"transformer.h.31.attn.masked_bias": "pytorch_model-00003-of-00003.bin",
|
| 363 |
+
"transformer.h.31.ln_1.bias": "pytorch_model-00003-of-00003.bin",
|
| 364 |
+
"transformer.h.31.ln_1.weight": "pytorch_model-00003-of-00003.bin",
|
| 365 |
+
"transformer.h.31.ln_2.bias": "pytorch_model-00003-of-00003.bin",
|
| 366 |
+
"transformer.h.31.ln_2.weight": "pytorch_model-00003-of-00003.bin",
|
| 367 |
+
"transformer.h.31.mlp.c_fc.bias": "pytorch_model-00003-of-00003.bin",
|
| 368 |
+
"transformer.h.31.mlp.c_fc.weight": "pytorch_model-00003-of-00003.bin",
|
| 369 |
+
"transformer.h.31.mlp.c_proj.bias": "pytorch_model-00003-of-00003.bin",
|
| 370 |
+
"transformer.h.31.mlp.c_proj.weight": "pytorch_model-00003-of-00003.bin",
|
| 371 |
+
"transformer.h.4.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 372 |
+
"transformer.h.4.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 373 |
+
"transformer.h.4.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 374 |
+
"transformer.h.4.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 375 |
+
"transformer.h.4.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 376 |
+
"transformer.h.4.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 377 |
+
"transformer.h.4.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 378 |
+
"transformer.h.4.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 379 |
+
"transformer.h.4.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 380 |
+
"transformer.h.4.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 381 |
+
"transformer.h.4.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 382 |
+
"transformer.h.4.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 383 |
+
"transformer.h.4.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 384 |
+
"transformer.h.4.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 385 |
+
"transformer.h.5.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 386 |
+
"transformer.h.5.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 387 |
+
"transformer.h.5.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 388 |
+
"transformer.h.5.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 389 |
+
"transformer.h.5.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 390 |
+
"transformer.h.5.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 391 |
+
"transformer.h.5.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 392 |
+
"transformer.h.5.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 393 |
+
"transformer.h.5.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 394 |
+
"transformer.h.5.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 395 |
+
"transformer.h.5.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 396 |
+
"transformer.h.5.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 397 |
+
"transformer.h.5.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 398 |
+
"transformer.h.5.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 399 |
+
"transformer.h.6.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 400 |
+
"transformer.h.6.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 401 |
+
"transformer.h.6.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 402 |
+
"transformer.h.6.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 403 |
+
"transformer.h.6.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 404 |
+
"transformer.h.6.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 405 |
+
"transformer.h.6.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 406 |
+
"transformer.h.6.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 407 |
+
"transformer.h.6.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 408 |
+
"transformer.h.6.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 409 |
+
"transformer.h.6.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 410 |
+
"transformer.h.6.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 411 |
+
"transformer.h.6.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 412 |
+
"transformer.h.6.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 413 |
+
"transformer.h.7.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 414 |
+
"transformer.h.7.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 415 |
+
"transformer.h.7.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 416 |
+
"transformer.h.7.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 417 |
+
"transformer.h.7.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 418 |
+
"transformer.h.7.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 419 |
+
"transformer.h.7.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 420 |
+
"transformer.h.7.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 421 |
+
"transformer.h.7.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 422 |
+
"transformer.h.7.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 423 |
+
"transformer.h.7.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 424 |
+
"transformer.h.7.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 425 |
+
"transformer.h.7.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 426 |
+
"transformer.h.7.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 427 |
+
"transformer.h.8.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 428 |
+
"transformer.h.8.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 429 |
+
"transformer.h.8.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 430 |
+
"transformer.h.8.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 431 |
+
"transformer.h.8.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 432 |
+
"transformer.h.8.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 433 |
+
"transformer.h.8.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 434 |
+
"transformer.h.8.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 435 |
+
"transformer.h.8.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 436 |
+
"transformer.h.8.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 437 |
+
"transformer.h.8.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 438 |
+
"transformer.h.8.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 439 |
+
"transformer.h.8.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 440 |
+
"transformer.h.8.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 441 |
+
"transformer.h.9.attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 442 |
+
"transformer.h.9.attn.c_attn.bias": "pytorch_model-00001-of-00003.bin",
|
| 443 |
+
"transformer.h.9.attn.c_attn.weight": "pytorch_model-00001-of-00003.bin",
|
| 444 |
+
"transformer.h.9.attn.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 445 |
+
"transformer.h.9.attn.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 446 |
+
"transformer.h.9.attn.masked_bias": "pytorch_model-00001-of-00003.bin",
|
| 447 |
+
"transformer.h.9.ln_1.bias": "pytorch_model-00001-of-00003.bin",
|
| 448 |
+
"transformer.h.9.ln_1.weight": "pytorch_model-00001-of-00003.bin",
|
| 449 |
+
"transformer.h.9.ln_2.bias": "pytorch_model-00001-of-00003.bin",
|
| 450 |
+
"transformer.h.9.ln_2.weight": "pytorch_model-00001-of-00003.bin",
|
| 451 |
+
"transformer.h.9.mlp.c_fc.bias": "pytorch_model-00001-of-00003.bin",
|
| 452 |
+
"transformer.h.9.mlp.c_fc.weight": "pytorch_model-00001-of-00003.bin",
|
| 453 |
+
"transformer.h.9.mlp.c_proj.bias": "pytorch_model-00001-of-00003.bin",
|
| 454 |
+
"transformer.h.9.mlp.c_proj.weight": "pytorch_model-00001-of-00003.bin",
|
| 455 |
+
"transformer.ln_f.bias": "pytorch_model-00003-of-00003.bin",
|
| 456 |
+
"transformer.ln_f.weight": "pytorch_model-00003-of-00003.bin",
|
| 457 |
+
"transformer.wpe.weight": "pytorch_model-00001-of-00003.bin",
|
| 458 |
+
"transformer.wte.weight": "pytorch_model-00001-of-00003.bin"
|
| 459 |
+
}
|
| 460 |
+
}
|
spiece.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f4bda2cd84ff0ac659cda40e746c55f47ee3e57cf18471670ad26998c28be52d
|
| 3 |
+
size 1071955
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_max_length": 2048,
|
| 3 |
+
"padding_side": "left",
|
| 4 |
+
"truncation_side": "left"
|
| 5 |
+
}
|