mk commited on
Commit
0c4d72d
1 Parent(s): 610fd06

add py file

Browse files
Files changed (49) hide show
  1. DATA_LICENSE +183 -0
  2. LICENSE +201 -0
  3. README.md +386 -0
  4. docker-compose.yml +28 -0
  5. lengths.ipynb +204 -0
  6. lora-alpaca-zh/README.md +20 -0
  7. lora-alpaca-zh/adapter_config.json +3 -0
  8. lora-alpaca-zh/adapter_model.bin +3 -0
  9. lora-alpaca-zh/checkpoint-1000/README.md +20 -0
  10. lora-alpaca-zh/checkpoint-1000/adapter_config.json +3 -0
  11. lora-alpaca-zh/checkpoint-1000/adapter_model.bin +3 -0
  12. lora-alpaca-zh/checkpoint-1000/optimizer.pt +3 -0
  13. lora-alpaca-zh/checkpoint-1000/rng_state.pth +3 -0
  14. lora-alpaca-zh/checkpoint-1000/scheduler.pt +3 -0
  15. lora-alpaca-zh/checkpoint-1000/trainer_state.json +3 -0
  16. lora-alpaca-zh/checkpoint-1000/training_args.bin +3 -0
  17. lora-alpaca-zh/checkpoint-600/README.md +20 -0
  18. lora-alpaca-zh/checkpoint-600/adapter_config.json +3 -0
  19. lora-alpaca-zh/checkpoint-600/adapter_model.bin +3 -0
  20. lora-alpaca-zh/checkpoint-600/optimizer.pt +3 -0
  21. lora-alpaca-zh/checkpoint-600/rng_state.pth +3 -0
  22. lora-alpaca-zh/checkpoint-600/scheduler.pt +3 -0
  23. lora-alpaca-zh/checkpoint-600/trainer_state.json +3 -0
  24. lora-alpaca-zh/checkpoint-600/training_args.bin +3 -0
  25. lora-alpaca-zh/checkpoint-800/README.md +20 -0
  26. lora-alpaca-zh/checkpoint-800/adapter_config.json +3 -0
  27. lora-alpaca-zh/checkpoint-800/adapter_model.bin +3 -0
  28. lora-alpaca-zh/checkpoint-800/optimizer.pt +3 -0
  29. lora-alpaca-zh/checkpoint-800/rng_state.pth +3 -0
  30. lora-alpaca-zh/checkpoint-800/scheduler.pt +3 -0
  31. lora-alpaca-zh/checkpoint-800/trainer_state.json +3 -0
  32. lora-alpaca-zh/checkpoint-800/training_args.bin +3 -0
  33. pyproject.toml +8 -0
  34. requirements.txt +12 -0
  35. templates/README.md +46 -0
  36. templates/alpaca.json +3 -0
  37. templates/alpaca_legacy.json +3 -0
  38. templates/alpaca_short.json +3 -0
  39. templates/vigogne.json +3 -0
  40. utils/README.md +13 -0
  41. utils/__init__.py +0 -0
  42. utils/__pycache__/__init__.cpython-311.pyc +0 -0
  43. utils/__pycache__/__init__.cpython-39.pyc +0 -0
  44. utils/__pycache__/callbacks.cpython-311.pyc +0 -0
  45. utils/__pycache__/callbacks.cpython-39.pyc +0 -0
  46. utils/__pycache__/prompter.cpython-311.pyc +0 -0
  47. utils/__pycache__/prompter.cpython-39.pyc +0 -0
  48. utils/callbacks.py +3 -0
  49. utils/prompter.py +3 -0
DATA_LICENSE ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Attribution License (ODC-By)
2
+ PREAMBLE
3
+ The Open Data Commons Attribution License is a license agreement intended to allow users to freely share, modify, and use this Database subject only to the attribution requirements set out in Section 4.
4
+
5
+ Databases can contain a wide variety of types of content (images, audiovisual material, and sounds all in the same database, for example), and so this license only governs the rights over the Database, and not the contents of the Database individually. Licensors may therefore wish to use this license together with another license for the contents.
6
+
7
+ Sometimes the contents of a database, or the database itself, can be covered by other rights not addressed here (such as private contracts, trademark over the name, or privacy rights / data protection rights over information in the contents), and so you are advised that you may have to consult other documents or clear other rights before doing activities not covered by this License.
8
+
9
+ The Licensor (as defined below)
10
+
11
+ and
12
+
13
+ You (as defined below)
14
+
15
+ agree as follows:
16
+
17
+ 1.0 DEFINITIONS OF CAPITALISED WORDS
18
+ “Collective Database” – Means this Database in unmodified form as part of a collection of independent databases in themselves that together are assembled into a collective whole. A work that constitutes a Collective Database will not be considered a Derivative Database.
19
+
20
+ “Convey” – As a verb, means Using the Database, a Derivative Database, or the Database as part of a Collective Database in any way that enables a Person to make or receive copies of the Database or a Derivative Database. Conveying does not include interaction with a user through a computer network, or creating and Using a Produced Work, where no transfer of a copy of the Database or a Derivative Database occurs.
21
+
22
+ “Contents” – The contents of this Database, which includes the information, independent works, or other material collected into the Database. For example, the contents of the Database could be factual data or works such as images, audiovisual material, text, or sounds.
23
+
24
+ “Database” – A collection of material (the Contents) arranged in a systematic or methodical way and individually accessible by electronic or other means offered under the terms of this License.
25
+
26
+ “Database Directive” – Means Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended or succeeded.
27
+
28
+ “Database Right” – Means rights resulting from the Chapter III (“sui generis”) rights in the Database Directive (as amended and as transposed by member states), which includes the Extraction and Re-utilisation of the whole or a Substantial part of the Contents, as well as any similar rights available in the relevant jurisdiction under Section 10.4.
29
+
30
+ “Derivative Database” – Means a database based upon the Database, and includes any translation, adaptation, arrangement, modification, or any other alteration of the Database or of a Substantial part of the Contents. This includes, but is not limited to, Extracting or Re-utilising the whole or a Substantial part of the Contents in a new Database.
31
+
32
+ “Extraction” – Means the permanent or temporary transfer of all or a Substantial part of the Contents to another medium by any means or in any form.
33
+
34
+ “License” – Means this license agreement and is both a license of rights such as copyright and Database Rights and an agreement in contract.
35
+
36
+ “Licensor” – Means the Person that offers the Database under the terms of this License.
37
+
38
+ “Person” – Means a natural or legal person or a body of persons corporate or incorporate.
39
+
40
+ “Produced Work” – a work (such as an image, audiovisual material, text, or sounds) resulting from using the whole or a Substantial part of the Contents (via a search or other query) from this Database, a Derivative Database, or this Database as part of a Collective Database.
41
+
42
+ “Publicly” – means to Persons other than You or under Your control by either more than 50% ownership or by the power to direct their activities (such as contracting with an independent consultant).
43
+
44
+ “Re-utilisation” – means any form of making available to the public all or a Substantial part of the Contents by the distribution of copies, by renting, by online or other forms of transmission.
45
+
46
+ “Substantial” – Means substantial in terms of quantity or quality or a combination of both. The repeated and systematic Extraction or Re-utilisation of insubstantial parts of the Contents may amount to the Extraction or Re-utilisation of a Substantial part of the Contents.
47
+
48
+ “Use” – As a verb, means doing any act that is restricted by copyright or Database Rights whether in the original medium or any other; and includes without limitation distributing, copying, publicly performing, publicly displaying, and preparing derivative works of the Database, as well as modifying the Database as may be technically necessary to use it in a different mode or format.
49
+
50
+ “You” – Means a Person exercising rights under this License who has not previously violated the terms of this License with respect to the Database, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation.
51
+
52
+ Words in the singular include the plural and vice versa.
53
+
54
+ 2.0 WHAT THIS LICENSE COVERS
55
+ 2.1. Legal effect of this document. This License is:
56
+
57
+ a. A license of applicable copyright and neighbouring rights;
58
+
59
+ b. A license of the Database Right; and
60
+
61
+ c. An agreement in contract between You and the Licensor.
62
+
63
+ 2.2 Legal rights covered. This License covers the legal rights in the Database, including:
64
+
65
+ a. Copyright. Any copyright or neighbouring rights in the Database. The copyright licensed includes any individual elements of the Database, but does not cover the copyright over the Contents independent of this Database. See Section 2.4 for details. Copyright law varies between jurisdictions, but is likely to cover: the Database model or schema, which is the structure, arrangement, and organisation of the Database, and can also include the Database tables and table indexes; the data entry and output sheets; and the Field names of Contents stored in the Database;
66
+
67
+ b. Database Rights. Database Rights only extend to the Extraction and Re-utilisation of the whole or a Substantial part of the Contents. Database Rights can apply even when there is no copyright over the Database. Database Rights can also apply when the Contents are removed from the Database and are selected and arranged in a way that would not infringe any applicable copyright; and
68
+
69
+ c. Contract. This is an agreement between You and the Licensor for access to the Database. In return you agree to certain conditions of use on this access as outlined in this License.
70
+
71
+ 2.3 Rights not covered.
72
+
73
+ a. This License does not apply to computer programs used in the making or operation of the Database;
74
+
75
+ b. This License does not cover any patents over the Contents or the Database; and
76
+
77
+ c. This License does not cover any trademarks associated with the Database.
78
+
79
+ 2.4 Relationship to Contents in the Database. The individual items of the Contents contained in this Database may be covered by other rights, including copyright, patent, data protection, privacy, or personality rights, and this License does not cover any rights (other than Database Rights or in contract) in individual Contents contained in the Database.
80
+
81
+ For example, if used on a Database of images (the Contents), this License would not apply to copyright over individual images, which could have their own separate licenses, or one single license covering all of the rights over the images.
82
+
83
+ 3.0 RIGHTS GRANTED
84
+ 3.1 Subject to the terms and conditions of this License, the Licensor grants to You a worldwide, royalty-free, non-exclusive, terminable (but only under Section 9) license to Use the Database for the duration of any applicable copyright and Database Rights. These rights explicitly include commercial use, and do not exclude any field of endeavour. To the extent possible in the relevant jurisdiction, these rights may be exercised in all media and formats whether now known or created in the future.
85
+
86
+ The rights granted cover, for example:
87
+
88
+ a. Extraction and Re-utilisation of the whole or a Substantial part of the Contents;
89
+
90
+ b. Creation of Derivative Databases;
91
+
92
+ c. Creation of Collective Databases;
93
+
94
+ d. Creation of temporary or permanent reproductions by any means and in any form, in whole or in part, including of any Derivative Databases or as a part of Collective Databases; and
95
+
96
+ e. Distribution, communication, display, lending, making available, or performance to the public by any means and in any form, in whole or in part, including of any Derivative Database or as a part of Collective Databases.
97
+
98
+ 3.2 Compulsory license schemes. For the avoidance of doubt:
99
+
100
+ a. Non-waivable compulsory license schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License;
101
+
102
+ b. Waivable compulsory license schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor waives the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; and,
103
+
104
+ c. Voluntary license schemes. The Licensor waives the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License.
105
+
106
+ 3.3 The right to release the Database under different terms, or to stop distributing or making available the Database, is reserved. Note that this Database may be multiple-licensed, and so You may have the choice of using alternative licenses for this Database. Subject to Section 10.4, all other rights not expressly granted by Licensor are reserved.
107
+
108
+ 4.0 CONDITIONS OF USE
109
+ 4.1 The rights granted in Section 3 above are expressly made subject to Your complying with the following conditions of use. These are important conditions of this License, and if You fail to follow them, You will be in material breach of its terms.
110
+
111
+ 4.2 Notices. If You Publicly Convey this Database, any Derivative Database, or the Database as part of a Collective Database, then You must:
112
+
113
+ a. Do so only under the terms of this License;
114
+
115
+ b. Include a copy of this License or its Uniform Resource Identifier (URI) with the Database or Derivative Database, including both in the Database or Derivative Database and in any relevant documentation;
116
+
117
+ c. Keep intact any copyright or Database Right notices and notices that refer to this License; and
118
+
119
+ d. If it is not possible to put the required notices in a particular file due to its structure, then You must include the notices in a location (such as a relevant directory) where users would be likely to look for it.
120
+
121
+ 4.3 Notice for using output (Contents). Creating and Using a Produced Work does not require the notice in Section 4.2. However, if you Publicly Use a Produced Work, You must include a notice associated with the Produced Work reasonably calculated to make any Person that uses, views, accesses, interacts with, or is otherwise exposed to the Produced Work aware that Content was obtained from the Database, Derivative Database, or the Database as part of a Collective Database, and that it is available under this License.
122
+
123
+ a. Example notice. The following text will satisfy notice under Section 4.3:
124
+
125
+ Contains information from DATABASE NAME which is made available
126
+ under the ODC Attribution License.
127
+ DATABASE NAME should be replaced with the name of the Database and a hyperlink to the location of the Database. “ODC Attribution License” should contain a hyperlink to the URI of the text of this License. If hyperlinks are not possible, You should include the plain text of the required URI’s with the above notice.
128
+
129
+ 4.4 Licensing of others. You may not sublicense the Database. Each time You communicate the Database, the whole or Substantial part of the Contents, or any Derivative Database to anyone else in any way, the Licensor offers to the recipient a license to the Database on the same terms and conditions as this License. You are not responsible for enforcing compliance by third parties with this License, but You may enforce any rights that You have over a Derivative Database. You are solely responsible for any modifications of a Derivative Database made by You or another Person at Your direction. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License.
130
+
131
+ 5.0 MORAL RIGHTS
132
+ 5.1 Moral rights. This section covers moral rights, including any rights to be identified as the author of the Database or to object to treatment that would otherwise prejudice the author’s honour and reputation, or any other derogatory treatment:
133
+
134
+ a. For jurisdictions allowing waiver of moral rights, Licensor waives all moral rights that Licensor may have in the Database to the fullest extent possible by the law of the relevant jurisdiction under Section 10.4;
135
+
136
+ b. If waiver of moral rights under Section 5.1 a in the relevant jurisdiction is not possible, Licensor agrees not to assert any moral rights over the Database and waives all claims in moral rights to the fullest extent possible by the law of the relevant jurisdiction under Section 10.4; and
137
+
138
+ c. For jurisdictions not allowing waiver or an agreement not to assert moral rights under Section 5.1 a and b, the author may retain their moral rights over certain aspects of the Database.
139
+
140
+ Please note that some jurisdictions do not allow for the waiver of moral rights, and so moral rights may still subsist over the Database in some jurisdictions.
141
+
142
+ 6.0 FAIR DEALING, DATABASE EXCEPTIONS, AND OTHER RIGHTS NOT AFFECTED
143
+ 6.1 This License does not affect any rights that You or anyone else may independently have under any applicable law to make any use of this Database, including without limitation:
144
+
145
+ a. Exceptions to the Database Right including: Extraction of Contents from non-electronic Databases for private purposes, Extraction for purposes of illustration for teaching or scientific research, and Extraction or Re-utilisation for public security or an administrative or judicial procedure.
146
+
147
+ b. Fair dealing, fair use, or any other legally recognised limitation or exception to infringement of copyright or other applicable laws.
148
+
149
+ 6.2 This License does not affect any rights of lawful users to Extract and Re-utilise insubstantial parts of the Contents, evaluated quantitatively or qualitatively, for any purposes whatsoever, including creating a Derivative Database (subject to other rights over the Contents, see Section 2.4). The repeated and systematic Extraction or Re-utilisation of insubstantial parts of the Contents may however amount to the Extraction or Re-utilisation of a Substantial part of the Contents.
150
+
151
+ 7.0 WARRANTIES AND DISCLAIMER
152
+ 7.1 The Database is licensed by the Licensor “as is” and without any warranty of any kind, either express, implied, or arising by statute, custom, course of dealing, or trade usage. Licensor specifically disclaims any and all implied warranties or conditions of title, non-infringement, accuracy or completeness, the presence or absence of errors, fitness for a particular purpose, merchantability, or otherwise. Some jurisdictions do not allow the exclusion of implied warranties, so this exclusion may not apply to You.
153
+
154
+ 8.0 LIMITATION OF LIABILITY
155
+ 8.1 Subject to any liability that may not be excluded or limited by law, the Licensor is not liable for, and expressly excludes, all liability for loss or damage however and whenever caused to anyone by any use under this License, whether by You or by anyone else, and whether caused by any fault on the part of the Licensor or not. This exclusion of liability includes, but is not limited to, any special, incidental, consequential, punitive, or exemplary damages such as loss of revenue, data, anticipated profits, and lost business. This exclusion applies even if the Licensor has been advised of the possibility of such damages.
156
+
157
+ 8.2 If liability may not be excluded by law, it is limited to actual and direct financial loss to the extent it is caused by proved negligence on the part of the Licensor.
158
+
159
+ 9.0 TERMINATION OF YOUR RIGHTS UNDER THIS LICENSE
160
+ 9.1 Any breach by You of the terms and conditions of this License automatically terminates this License with immediate effect and without notice to You. For the avoidance of doubt, Persons who have received the Database, the whole or a Substantial part of the Contents, Derivative Databases, or the Database as part of a Collective Database from You under this License will not have their licenses terminated provided their use is in full compliance with this License or a license granted under Section 4.8 of this License. Sections 1, 2, 7, 8, 9 and 10 will survive any termination of this License.
161
+
162
+ 9.2 If You are not in breach of the terms of this License, the Licensor will not terminate Your rights under it.
163
+
164
+ 9.3 Unless terminated under Section 9.1, this License is granted to You for the duration of applicable rights in the Database.
165
+
166
+ 9.4 Reinstatement of rights. If you cease any breach of the terms and conditions of this License, then your full rights under this License will be reinstated:
167
+
168
+ a. Provisionally and subject to permanent termination until the 60th day after cessation of breach;
169
+
170
+ b. Permanently on the 60th day after cessation of breach unless otherwise reasonably notified by the Licensor; or
171
+
172
+ c. Permanently if reasonably notified by the Licensor of the violation, this is the first time You have received notice of violation of this License from the Licensor, and You cure the violation prior to 30 days after your receipt of the notice.
173
+
174
+ 9.5 Notwithstanding the above, Licensor reserves the right to release the Database under different license terms or to stop distributing or making available the Database. Releasing the Database under different license terms or stopping the distribution of the Database will not withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above.
175
+
176
+ 10.0 GENERAL
177
+ 10.1 If any provision of this License is held to be invalid or unenforceable, that must not affect the validity or enforceability of the remainder of the terms and conditions of this License and each remaining provision of this License shall be valid and enforced to the fullest extent permitted by law.
178
+
179
+ 10.2 This License is the entire agreement between the parties with respect to the rights granted here over the Database. It replaces any earlier understandings, agreements or representations with respect to the Database.
180
+
181
+ 10.3 If You are in breach of the terms of this License, You will not be entitled to rely on the terms of this License or to complain of any breach by the Licensor.
182
+
183
+ 10.4 Choice of law. This License takes effect in and will be governed by the laws of the relevant jurisdiction in which the License terms are sought to be enforced. If the standard suite of rights granted under applicable copyright law and Database Rights in the relevant jurisdiction includes additional rights not granted under this License, these additional rights are granted in this License in order to meet the terms of this License.
LICENSE ADDED
@@ -0,0 +1,201 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright 2023 Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
README.md ADDED
@@ -0,0 +1,386 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🦙🌲🤏 Alpaca-LoRA
2
+
3
+ - 🤗 **Try the pretrained model out [here](https://huggingface.co/spaces/tloen/alpaca-lora), courtesy of a GPU grant from Huggingface!**
4
+ - Users have created a Discord server for discussion and support [here](https://discord.gg/prbq284xX5)
5
+ - 4/14: Chansung Park's GPT4-Alpaca adapters: https://github.com/tloen/alpaca-lora/issues/340
6
+
7
+ This repository contains code for reproducing the [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) results using [low-rank adaptation (LoRA)](https://arxiv.org/pdf/2106.09685.pdf).
8
+ We provide an Instruct model of similar quality to `text-davinci-003` that can run [on a Raspberry Pi](https://twitter.com/miolini/status/1634982361757790209) (for research),
9
+ and the code is easily extended to the `13b`, `30b`, and `65b` models.
10
+
11
+ In addition to the training code, which runs within hours on a single RTX 4090,
12
+ we publish a script for downloading and inference on the foundation model and LoRA,
13
+ as well as the resulting [LoRA weights themselves](https://huggingface.co/tloen/alpaca-lora-7b/tree/main).
14
+ To fine-tune cheaply and efficiently, we use Hugging Face's [PEFT](https://github.com/huggingface/peft)
15
+ as well as Tim Dettmers' [bitsandbytes](https://github.com/TimDettmers/bitsandbytes).
16
+
17
+ Without hyperparameter tuning, the LoRA model produces outputs comparable to the Stanford Alpaca model. (Please see the outputs included below.) Further tuning might be able to achieve better performance; I invite interested users to give it a try and report their results.
18
+
19
+ ### Local Setup
20
+
21
+ 1. Install dependencies
22
+
23
+ ```bash
24
+ pip install -r requirements.txt
25
+ ```
26
+
27
+ 1. If bitsandbytes doesn't work, [install it from source.](https://github.com/TimDettmers/bitsandbytes/blob/main/compile_from_source.md) Windows users can follow [these instructions](https://github.com/tloen/alpaca-lora/issues/17).
28
+
29
+ ### Training (`finetune.py`)
30
+
31
+ This file contains a straightforward application of PEFT to the LLaMA model,
32
+ as well as some code related to prompt construction and tokenization.
33
+ PRs adapting this code to support larger models are always welcome.
34
+
35
+ Example usage:
36
+
37
+ ```bash
38
+ python finetune.py \
39
+ --base_model 'decapoda-research/llama-7b-hf' \
40
+ --data_path 'yahma/alpaca-cleaned' \
41
+ --output_dir './lora-alpaca'
42
+ ```
43
+
44
+ We can also tweak our hyperparameters:
45
+
46
+ ```bash
47
+ python finetune.py \
48
+ --base_model 'decapoda-research/llama-7b-hf' \
49
+ --data_path 'yahma/alpaca-cleaned' \
50
+ --output_dir './lora-alpaca' \
51
+ --batch_size 128 \
52
+ --micro_batch_size 4 \
53
+ --num_epochs 3 \
54
+ --learning_rate 1e-4 \
55
+ --cutoff_len 512 \
56
+ --val_set_size 2000 \
57
+ --lora_r 8 \
58
+ --lora_alpha 16 \
59
+ --lora_dropout 0.05 \
60
+ --lora_target_modules '[q_proj,v_proj]' \
61
+ --train_on_inputs \
62
+ --group_by_length
63
+ ```
64
+
65
+ ### Inference (`generate.py`)
66
+
67
+ This file reads the foundation model from the Hugging Face model hub and the LoRA weights from `tloen/alpaca-lora-7b`, and runs a Gradio interface for inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.
68
+
69
+ Example usage:
70
+
71
+ ```bash
72
+ python generate.py \
73
+ --load_8bit \
74
+ --base_model 'decapoda-research/llama-7b-hf' \
75
+ --lora_weights 'tloen/alpaca-lora-7b'
76
+ ```
77
+
78
+ ### Official weights
79
+
80
+ The most recent "official" Alpaca-LoRA adapter available at [`tloen/alpaca-lora-7b`](https://huggingface.co/tloen/alpaca-lora-7b) was trained on March 26 with the following command:
81
+
82
+ ```bash
83
+ python finetune.py \
84
+ --base_model='decapoda-research/llama-7b-hf' \
85
+ --num_epochs=10 \
86
+ --cutoff_len=512 \
87
+ --group_by_length \
88
+ --output_dir='./lora-alpaca' \
89
+ --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
90
+ --lora_r=16 \
91
+ --micro_batch_size=8
92
+ ```
93
+
94
+ ### Checkpoint export (`export_*_checkpoint.py`)
95
+
96
+ These files contain scripts that merge the LoRA weights back into the base model
97
+ for export to Hugging Face format and to PyTorch `state_dicts`.
98
+ They should help users
99
+ who want to run inference in projects like [llama.cpp](https://github.com/ggerganov/llama.cpp)
100
+ or [alpaca.cpp](https://github.com/antimatter15/alpaca.cpp).
101
+
102
+ ### Docker Setup & Inference
103
+
104
+ 1. Build the container image:
105
+
106
+ ```bash
107
+ docker build -t alpaca-lora .
108
+ ```
109
+
110
+ 2. Run the container (you can also use `finetune.py` and all of its parameters as shown above for training):
111
+
112
+ ```bash
113
+ docker run --gpus=all --shm-size 64g -p 7860:7860 -v ${HOME}/.cache:/root/.cache --rm alpaca-lora generate.py \
114
+ --load_8bit \
115
+ --base_model 'decapoda-research/llama-7b-hf' \
116
+ --lora_weights 'tloen/alpaca-lora-7b'
117
+ ```
118
+
119
+ 3. Open `https://localhost:7860` in the browser
120
+
121
+ ### Docker Compose Setup & Inference
122
+
123
+ 1. (optional) Change desired model and weights under `environment` in the `docker-compose.yml`
124
+
125
+ 2. Build and run the container
126
+
127
+ ```bash
128
+ docker-compose up -d --build
129
+ ```
130
+
131
+ 3. Open `https://localhost:7860` in the browser
132
+
133
+ 4. See logs:
134
+
135
+ ```bash
136
+ docker-compose logs -f
137
+ ```
138
+
139
+ 5. Clean everything up:
140
+
141
+ ```bash
142
+ docker-compose down --volumes --rmi all
143
+ ```
144
+
145
+ ### Notes
146
+
147
+ - We can likely improve our model performance significantly if we had a better dataset. Consider supporting the [LAION Open Assistant](https://open-assistant.io/) effort to produce a high-quality dataset for supervised fine-tuning (or bugging them to release their data).
148
+ - We're continually fixing bugs and conducting training runs, and the weights on the Hugging Face Hub are being updated accordingly. In particular, those facing issues with response lengths should make sure that they have the latest version of the weights and code.
149
+ - Users with multiple GPUs should take a look [here](https://github.com/tloen/alpaca-lora/issues/8#issuecomment-1477490259).
150
+ - We include the Stanford Alpaca dataset, which was made available under the ODC Attribution License.
151
+
152
+ ### Resources
153
+
154
+ - [alpaca.cpp](https://github.com/antimatter15/alpaca.cpp), a native client for running Alpaca models on the CPU
155
+ - [Alpaca-LoRA-Serve](https://github.com/deep-diver/Alpaca-LoRA-Serve), a ChatGPT-style interface for Alpaca models
156
+ - [AlpacaDataCleaned](https://github.com/gururise/AlpacaDataCleaned), a project to improve the quality of the Alpaca dataset
157
+ - [GPT-4 Alpaca Data](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM) a project to port synthetic data creation to GPT-4
158
+ - [dolly-15k-instruction-alpaca-format](https://huggingface.co/datasets/c-s-ale/dolly-15k-instruction-alpaca-format), an Alpaca-compatible version of [Databricks' Dolly 15k human-generated instruct dataset](https://github.com/databrickslabs/dolly/tree/master/data) (see [blog](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm))
159
+ - [Alpaca-LoRA MT](https://github.com/juletx/alpaca-lora-mt), a project to finetune models with [machine-translated Alpaca data](https://huggingface.co/datasets/HiTZ/alpaca_mt) in 6 Iberian languages: Portuguese, Spanish, Catalan, Basque, Galician and Asturian.
160
+ - Various adapter weights (download at own risk):
161
+ - 7B:
162
+ - 3️⃣ <https://huggingface.co/tloen/alpaca-lora-7b>
163
+ - 3️⃣ <https://huggingface.co/samwit/alpaca7B-lora>
164
+ - **4️⃣ <https://huggingface.co/chansung/gpt4-alpaca-lora-7b>**
165
+ - 🚀 <https://huggingface.co/nomic-ai/gpt4all-lora>
166
+ - 🇧🇷 <https://huggingface.co/22h/cabrita-lora-v0-1>
167
+ - 🇨🇳 <https://huggingface.co/qychen/luotuo-lora-7b-0.1>
168
+ - 🇨🇳 <https://huggingface.co/ziqingyang/chinese-alpaca-lora-7b>
169
+ - 🇯🇵 <https://huggingface.co/kunishou/Japanese-Alapaca-LoRA-7b-v0>
170
+ - 🇫🇷 <https://huggingface.co/bofenghuang/vigogne-lora-7b>
171
+ - 🇹🇭 <https://huggingface.co/Thaweewat/thai-buffala-lora-7b-v0-1>
172
+ - 🇩🇪 <https://huggingface.co/thisserand/alpaca_lora_german>
173
+ - 🇵🇱 <https://huggingface.co/mmosiolek/polpaca-lora-7b>
174
+ - 🇵🇱 <https://huggingface.co/chrisociepa/alpaca-lora-7b-pl>
175
+ - 🇮🇹 <https://huggingface.co/teelinsan/camoscio-7b-llama>
176
+ - 🇷🇺 <https://huggingface.co/IlyaGusev/llama_7b_ru_turbo_alpaca_lora>
177
+ - 🇺🇦 <https://huggingface.co/robinhad/ualpaca-7b-llama>
178
+ - 🇮🇹 <https://huggingface.co/mchl-labs/stambecco-7b-plus>
179
+ - 🇪🇸 <https://huggingface.co/plncmm/guanaco-lora-7b>
180
+ - 🇬🇧 🇪🇸 🇵🇹 <https://huggingface.co/HiTZ/alpaca-lora-7b-en-pt-es-ca-eu-gl-at>
181
+ - 13B:
182
+ - 3️⃣ <https://huggingface.co/Angainor/alpaca-lora-13b>
183
+ - 3️⃣ <https://huggingface.co/chansung/alpaca-lora-13b>
184
+ - 3️⃣ <https://huggingface.co/mattreid/alpaca-lora-13b>
185
+ - 3️⃣ <https://huggingface.co/samwit/alpaca13B-lora>
186
+ - **4️⃣ <https://huggingface.co/chansung/gpt4-alpaca-lora-13b>**
187
+ - 🇯🇵 <https://huggingface.co/kunishou/Japanese-Alapaca-LoRA-13b-v0>
188
+ - 🇰🇷 <https://huggingface.co/chansung/koalpaca-lora-13b>
189
+ - 🇨🇳 <https://huggingface.co/facat/alpaca-lora-cn-13b>
190
+ - 🇨🇳 <https://huggingface.co/ziqingyang/chinese-alpaca-lora-13b>
191
+ - 🇪🇸 <https://huggingface.co/plncmm/guanaco-lora-13b>
192
+ - 🇮🇹 <https://huggingface.co/mchl-labs/stambecco-13b-plus>
193
+ - 🇬🇧 🇪🇸 🇵🇹 <https://huggingface.co/HiTZ/alpaca-lora-13b-en-pt-es-ca-eu-gl-at>
194
+ - 30B:
195
+ - 3️⃣ <https://huggingface.co/baseten/alpaca-30b>
196
+ - 3️⃣ <https://huggingface.co/chansung/alpaca-lora-30b>
197
+ - **4️⃣ <https://huggingface.co/chansung/gpt4-alpaca-lora-30b>**
198
+ - 🇯🇵 <https://huggingface.co/kunishou/Japanese-Alapaca-LoRA-30b-v0>
199
+ - 🇬🇧 🇪🇸 🇵🇹 <https://huggingface.co/HiTZ/alpaca-lora-30b-en-pt-es-ca-eu-gl-at>
200
+ - 65B
201
+ - <https://huggingface.co/chansung/alpaca-lora-65b>
202
+ - 🇬🇧 🇪🇸 🇵🇹 <https://huggingface.co/HiTZ/alpaca-lora-65b-en-pt-es-ca>
203
+ - [alpaca-native](https://huggingface.co/chavinlo/alpaca-native), a replication using the original Alpaca code
204
+ - [llama.onnx](https://github.com/tpoisonooo/llama.onnx), a project to inference alpaca with onnx format
205
+
206
+ ### Example outputs
207
+
208
+ **Instruction**: Tell me about alpacas.
209
+
210
+ **Alpaca-LoRA**: Alpacas are members of the camelid family and are native to the Andes Mountains of South America. They are known for their soft, luxurious fleece, which is used to make clothing, blankets, and other items. Alpacas are herbivores and graze on grasses and other plants. They are social animals and live in herds of up to 20 individuals.
211
+
212
+ **Stanford Alpaca**: Alpacas are small, fluffy animals related to camels and llamas. They are native to Peru and Bolivia, and were first domesticated around 5,000 years ago. They are kept mainly for their fine, soft fleece, which is used to make knitwear and other garments. Alpacas are herd animals and live in small family groups, led by an older male. They are highly social animals and can form strong bonds with their owners.
213
+
214
+ **text-davinci-003**: Alpacas are a domesticated species of South American camelid, similar to llamas. They are native to the Andes Mountains in South America and are kept as livestock for their soft, luxurious wool. Alpacas are social animals and live in herds of up to 20 individuals. They are typically kept in small herds of two to five animals, and are relatively easy to care for. Alpacas are herbivores and feed on grass, hay, and other vegetation. They are also known for their gentle and friendly nature, making them popular as pets.
215
+
216
+ ---
217
+
218
+ **Instruction**: Tell me about the president of Mexico in 2019.
219
+
220
+ **Alpaca-LoRA**: The president of Mexico in 2019 was Andrés Manuel López Obrador, who took office on December 1, 2018. He is a member of the National Regeneration Movement (MORENA) political party and is the first left-wing president of Mexico since 1946. He is known for his anti-corruption and anti-neoliberal policies, as well as his commitment to improving the living conditions of the Mexican people.
221
+
222
+ **Stanford Alpaca**: In 2019, Andres Manuel Lopez Obrador was sworn in as President of Mexico. He is the country’s 58th president and the first from the left-wing Morena party. He was elected in 2018 on a promise to combat corruption and reduce poverty and has faced criticism for his handling of the economy, crime and the country’s relationship with the United States. He is also known for his ambitious social welfare programs and has been praised for raising the minimum wage and providing aid to low-income families. He is seen as a champion of democracy and human rights in Mexico.
223
+
224
+ **text-davinci-003**: The president of Mexico in 2019 is Andrés Manuel López Obrador, commonly known as AMLO. He is a Mexican politician who has been the President of Mexico since December 1, 2018. He is a member of the National Regeneration Movement (MORENA) party and previously served as the Head of Government of Mexico City from 2000 to 2005. He is a left-wing populist and has promised to reduce corruption and inequality in Mexico. He has also promised to increase public spending on social programs and infrastructure projects.
225
+
226
+ ---
227
+
228
+ **Instruction**: Tell me about the king of France in 2019.
229
+
230
+ **Alpaca-LoRA**: The King of France in 2019 is Emmanuel Macron, who was elected in May 2017. He is the youngest president in the history of the Fifth Republic and the first president to be born after World War II. He is also the first president to have never held elected office before.
231
+
232
+ **Stanford Alpaca**: In 2019, the King of France was Emmanuel Macron. He was sworn in as President of France on May 14, 2017, and has been leading the country ever since. He is a member of the centrist La République En Marche! Party. He is known for his reformist agenda and has faced a number of protests from both the right and left of the political spectrum. He is the youngest French president in history.
233
+
234
+ **text-davinci-003**: The King of France is a largely symbolic role, as France is a republic. The current President of France is Emmanuel Macron, who was elected in 2017.
235
+
236
+ ---
237
+
238
+ **Instruction**: List all Canadian provinces in alphabetical order.
239
+
240
+ **Alpaca-LoRA**: Alberta, British Columbia, Manitoba, New Brunswick, Newfoundland and Labrador, Nova Scotia, Ontario, Prince Edward Island, Quebec, Saskatchewan
241
+
242
+ **Stanford Alpaca**: Alberta, British Columbia, Manitoba, New Brunswick, Newfoundland and Labrador, Nova Scotia, Ontario, Prince Edward Island, Quebec, Saskatchewan, Northwest Territories, Nunavut, Yukon.
243
+
244
+ **text-davinci-003**:
245
+
246
+ 1. Alberta
247
+ 2. British Columbia
248
+ 3. Manitoba
249
+ 4. New Brunswick
250
+ 5. Newfoundland and Labrador
251
+ 6. Northwest Territories
252
+ 7. Nova Scotia
253
+ 8. Nunavut
254
+ 9. Ontario
255
+ 10. Prince Edward Island
256
+ 11. Quebec
257
+ 12. Saskatchewan
258
+ 13. Yukon
259
+
260
+ ---
261
+
262
+ **Instruction**: Write a Python program that prints the first 10 Fibonacci numbers.
263
+
264
+ **Alpaca-LoRA**:
265
+
266
+ ```python
267
+ def fibonacci(n):
268
+ if n == 0:
269
+ return 0
270
+ elif n == 1:
271
+ return 1
272
+ else:
273
+ return fibonacci(n-1) + fibonacci(n-2)
274
+
275
+ print(fibonacci(10))
276
+ ```
277
+
278
+ **Stanford Alpaca**:
279
+
280
+ ```python
281
+ numbers = [0, 1]
282
+
283
+ for i in range(2, 11):
284
+ numbers.append(numbers[i-2] + numbers[i-1])
285
+
286
+ print(numbers)
287
+ # Output: [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
288
+ ```
289
+
290
+ **text-davinci-003**:
291
+
292
+ ```python
293
+ # Fibonacci numbers are the numbers in the following integer sequence, called the Fibonacci sequence, and characterized by the fact that every number after the first two is the sum of the two preceding ones.
294
+
295
+ # Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...
296
+
297
+ # Program:
298
+
299
+ # Initialize the first two numbers of the Fibonacci sequence
300
+ a = 0
301
+ b = 1
302
+
303
+ # Print the first 10 numbers of the Fibonacci sequence
304
+ print(a)
305
+ print(b)
306
+
307
+ for i in range(2,10):
308
+ c = a + b
309
+ a = b
310
+ b = c
311
+ print(c)
312
+ ```
313
+
314
+ ---
315
+
316
+ **Instruction**: Write a program that prints the numbers from 1 to 100. But for multiples of three print 'Fizz' instead of the number and for the multiples of five print 'Buzz'. For numbers which are multiples of both three and five print 'FizzBuzz'.
317
+
318
+ **Alpaca-LoRA**:
319
+
320
+ ```python
321
+ for i in range(1, 101):
322
+ if i % 3 == 0 and i % 5 == 0:
323
+ print("FizzBuzz")
324
+ elif i % 3 == 0:
325
+ print("Fizz")
326
+ elif i % 5 == 0:
327
+ print("Buzz")
328
+ else:
329
+ print(i)
330
+ ```
331
+
332
+ **Stanford Alpaca**:
333
+
334
+ ```python
335
+ for i in range(1, 101):
336
+ if i % 3 == 0 and i % 5 == 0:
337
+ print('FizzBuzz')
338
+ elif i % 3 == 0:
339
+ print('Fizz')
340
+ elif i % 5 == 0:
341
+ print('Buzz')
342
+ else:
343
+ print(i)
344
+ ```
345
+
346
+ **text-davinci-003**:
347
+
348
+ ```javascript
349
+ for (let i = 1; i <= 100; i++) {
350
+ if (i % 3 === 0 && i % 5 === 0) {
351
+ console.log("FizzBuzz");
352
+ } else if (i % 3 === 0) {
353
+ console.log("Fizz");
354
+ } else if (i % 5 === 0) {
355
+ console.log("Buzz");
356
+ } else {
357
+ console.log(i);
358
+ }
359
+ }
360
+ ```
361
+
362
+ ---
363
+
364
+ **Instruction**: Tell me five words that rhyme with 'shock'.
365
+
366
+ **Alpaca-LoRA**: Flock, lock, rock, stock, and sock.
367
+
368
+ **Stanford Alpaca**: Five words that rhyme with shock are: rock, pop, shock, cook, and snock.
369
+
370
+ **text-davinci-003**:
371
+
372
+ 1. Block
373
+ 2. Lock
374
+ 3. Dock
375
+ 4. Clock
376
+ 5. Rock
377
+
378
+ ---
379
+
380
+ **Instruction**: Translate the sentence 'I have no mouth but I must scream' into Spanish.
381
+
382
+ **Alpaca-LoRA**: No tengo boca pero tengo que gritar.
383
+
384
+ **Stanford Alpaca**: No tengo boca, pero debo gritar.
385
+
386
+ **text-davinci-003**: No tengo boca pero debo gritar.
docker-compose.yml ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ version: '3'
2
+
3
+ services:
4
+ alpaca-lora:
5
+ build:
6
+ context: ./
7
+ dockerfile: Dockerfile
8
+ args:
9
+ BUILDKIT_INLINE_CACHE: "0"
10
+ image: alpaca-lora
11
+ shm_size: '64gb'
12
+ command: generate.py --load_8bit --base_model $BASE_MODEL --lora_weights 'tloen/alpaca-lora-7b'
13
+ restart: unless-stopped
14
+ volumes:
15
+ - alpaca-lora:/root/.cache # Location downloaded weights will be stored
16
+ ports:
17
+ - 7860:7860
18
+ deploy:
19
+ resources:
20
+ reservations:
21
+ devices:
22
+ - driver: nvidia
23
+ count: all
24
+ capabilities: [ gpu ]
25
+
26
+ volumes:
27
+ alpaca-lora:
28
+ name: alpaca-lora
lengths.ipynb ADDED
@@ -0,0 +1,204 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cells": [
3
+ {
4
+ "cell_type": "code",
5
+ "execution_count": 1,
6
+ "metadata": {},
7
+ "outputs": [
8
+ {
9
+ "name": "stderr",
10
+ "output_type": "stream",
11
+ "text": [
12
+ "/home/eric/miniconda3/envs/dl3/lib/python3.10/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
13
+ " from .autonotebook import tqdm as notebook_tqdm\n",
14
+ "Found cached dataset json (/home/eric/.cache/huggingface/datasets/json/default-789f51900889f651/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51)\n",
15
+ "100%|██████████| 1/1 [00:00<00:00, 784.28it/s]\n",
16
+ "Loading cached processed dataset at /home/eric/.cache/huggingface/datasets/json/default-789f51900889f651/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-f691ee34ec2034cb.arrow\n"
17
+ ]
18
+ }
19
+ ],
20
+ "source": [
21
+ "from datasets import load_dataset\n",
22
+ "from transformers import LlamaTokenizer\n",
23
+ "\n",
24
+ "\n",
25
+ "tokenizer = LlamaTokenizer.from_pretrained(\n",
26
+ " \"decapoda-research/llama-7b-hf\", add_eos_token=True\n",
27
+ ")\n",
28
+ "tokenizer.pad_token = tokenizer.eos_token\n",
29
+ "tokenizer.pad_token_id = tokenizer.eos_token_id\n",
30
+ "\n",
31
+ "data = load_dataset(\"json\", data_files=\"alpaca_data.json\")\n",
32
+ "\n",
33
+ "\n",
34
+ "def generate_prompt(data_point):\n",
35
+ " # sorry about the formatting disaster gotta move fast\n",
36
+ " if data_point[\"input\"]:\n",
37
+ " return f\"\"\"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n",
38
+ "\n",
39
+ "### Instruction:\n",
40
+ "{data_point[\"instruction\"]}\n",
41
+ "\n",
42
+ "### Input:\n",
43
+ "{data_point[\"input\"]}\n",
44
+ "\n",
45
+ "### Response:\n",
46
+ "{data_point[\"output\"]}\"\"\"\n",
47
+ " else:\n",
48
+ " return f\"\"\"Below is an instruction that describes a task. Write a response that appropriately completes the request.\n",
49
+ "\n",
50
+ "### Instruction:\n",
51
+ "{data_point[\"instruction\"]}\n",
52
+ "\n",
53
+ "### Response:\n",
54
+ "{data_point[\"output\"]}\"\"\"\n",
55
+ "\n",
56
+ "\n",
57
+ "data = data.map(\n",
58
+ " lambda data_point: {\"prompt\": tokenizer(generate_prompt(data_point))}\n",
59
+ ")"
60
+ ]
61
+ },
62
+ {
63
+ "cell_type": "code",
64
+ "execution_count": 2,
65
+ "metadata": {},
66
+ "outputs": [
67
+ {
68
+ "data": {
69
+ "text/plain": [
70
+ "<matplotlib.lines.Line2D at 0x7f6f1af20af0>"
71
+ ]
72
+ },
73
+ "execution_count": 2,
74
+ "metadata": {},
75
+ "output_type": "execute_result"
76
+ },
77
+ {
78
+ "data": {
79
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGzCAYAAAAxPS2EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/q0lEQVR4nO3de1hVZf7//xeobBDcG0+AJCqTjkoeSizd09Ekyeio1mhmaJpfDSu0MbUaM5vS0UrtoFbOiJ/ppHaVleSBxMOUpEZRamk6YTgZWBls8QAK9++PfqxxCyoQCAuej+va18W+13uvfa8bkJf3WvfaPsYYIwAAABvxrekOAAAAVBQBBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBvXWtGnT5OPjc17e65prrtE111xjPd+wYYN8fHz09ttvn5f3Hz58uNq1a3de3quy8vPzNWrUKIWFhcnHx0eJiYk13aVareRnaMOGDTXdlXPy8fHRuHHjarobqGMIMKgTkpKS5OPjYz38/f0VHh6u2NhYPf/88zp8+HCVvM+BAwc0bdo0ZWRkVMn+qlJt7lt5PP3000pKStLYsWP1r3/9S8OGDavpLp1XR48e1bRp02wRSMqyefNmTZs2Tbm5uTXdFdQTDWu6A0BVmj59uiIjI3XixAllZ2drw4YNSkxM1HPPPaf3339f3bp1s2ofe+wxTZ48uUL7P3DggJ544gm1a9dOF198cblft3bt2gq9T2WcrW+vvvqqiouLq70Pv0dqaqp69+6txx9/vKa7UiOOHj2qJ554QpK8ZuvsYvPmzXriiSc0fPhwBQcH13R3UA8QYFCn9O/fXz179rSeT5kyRampqbrxxht1880365tvvlFAQIAkqWHDhmrYsHp/BY4eParGjRvLz8+vWt/nXBo1alSj718eBw8eVFRUVLW/z5EjRxQYGFjt7wOgenEKCXXetddeq7/+9a/6/vvv9dprr1ntZV0Dk5KSoiuuuELBwcEKCgpSx44d9cgjj0j67ZqDSy+9VJI0YsQI63RVUlKSpN/+19ylSxelp6frqquuUuPGja3Xnn4NTImioiI98sgjCgsLU2BgoG6++Wbt37/fq6Zdu3YaPnx4qdeeus9z9a2sa2COHDmihx56SBEREXI4HOrYsaOeeeYZnf4B9SXXL6xYsUJdunSRw+HQRRddpNWrV5c94Kc5ePCgRo4cqdDQUPn7+6t79+5asmSJtb3kWo7MzEwlJydbfd+3b98Z91nSp9dff10dO3aUv7+/oqOjtWnTJq+6ku/x119/rTvvvFNNmzbVFVdcIUk6efKknnzySV144YVyOBxq166dHnnkERUUFHjto127drrxxhu1YcMG9ezZUwEBAeratat1quedd95R165drT588cUXXq8fPny4goKC9N133yk2NlaBgYEKDw/X9OnTrbHet2+fWrZsKUl64oknrDGYNm1aucb4VFu2bNH1118vl8ulxo0b6+qrr9Ynn3xS5rjs3bvXmjFxuVwaMWKEjh496lV77NgxPfDAA2rRooWaNGmim2++WT/88INX/6ZNm6aJEydKkiIjI8/4PTzXz9Dhw4eVmJiodu3ayeFwKCQkRNddd50+//zzCo8D6j5mYFAvDBs2TI888ojWrl2re++9t8yanTt36sYbb1S3bt00ffp0ORwO7d271/rHv3Pnzpo+fbqmTp2q0aNH68orr5Qk/elPf7L28csvv6h///4aPHiw7rrrLoWGhp61X0899ZR8fHw0adIkHTx4UHPnzlVMTIwyMjKsmaLyKE/fTmWM0c0336z169dr5MiRuvjii7VmzRpNnDhRP/zwg+bMmeNV//HHH+udd97RfffdpyZNmuj555/XwIEDlZWVpebNm5+xX8eOHdM111yjvXv3aty4cYqMjNTy5cs1fPhw5ebm6sEHH1Tnzp31r3/9S+PHj1fr1q310EMPSZL1B/1MNm7cqKVLl+qBBx6Qw+HQ/Pnzdf3112vr1q3q0qWLV+3tt9+uDh066Omnn7ZCw6hRo7RkyRINGjRIDz30kLZs2aIZM2bom2++0bvvvuv1+r179+rOO+/U//t//0933XWXnnnmGd10001auHChHnnkEd13332SpBkzZuiOO+7Q7t275ev7v/8fFhUV6frrr1fv3r01a9YsrV69Wo8//rhOnjyp6dOnq2XLllqwYIHGjh2r2267TQMGDJAkr1Oe5ZGamqr+/fsrOjpajz/+uHx9fbV48WJde+21+ve//63LLrvMq/6OO+5QZGSkZsyYoc8//1yLFi1SSEiI/v73v1s1w4cP17JlyzRs2DD17t1bGzduVFxcnNd+BgwYoG+//VZvvvmm5syZoxYtWkjy/h6W52dozJgxevvttzVu3DhFRUXpl19+0ccff6xvvvlGPXr0qNBYoB4wQB2wePFiI8ls27btjDUul8tccskl1vPHH3/cnPorMGfOHCPJ/PTTT2fcx7Zt24wks3jx4lLbrr76aiPJLFy4sMxtV199tfV8/fr1RpK54IILjMfjsdqXLVtmJJl58+ZZbW3btjXx8fHn3OfZ+hYfH2/atm1rPV+xYoWRZP72t7951Q0aNMj4+PiYvXv3Wm2SjJ+fn1fbl19+aSSZF154odR7nWru3LlGknnttdestsLCQuN2u01QUJDXsbdt29bExcWddX+n9kmS+eyzz6y277//3vj7+5vbbrvNaiv5Hg8ZMsTr9RkZGUaSGTVqlFf7X/7yFyPJpKamevVLktm8ebPVtmbNGiPJBAQEmO+//95qf/nll40ks379eqstPj7eSDL333+/1VZcXGzi4uKMn5+f9fP2008/GUnm8ccfL9cYlPwMlbxXcXGx6dChg4mNjTXFxcVW3dGjR01kZKS57rrrSo3LPffc47XP2267zTRv3tx6np6ebiSZxMREr7rhw4eX6uvs2bONJJOZmVmqr+X9GXK5XCYhIaFcxw9wCgn1RlBQ0FlXI5VcePjee+9V+oJXh8OhESNGlLv+7rvvVpMmTazngwYNUqtWrfThhx9W6v3L68MPP1SDBg30wAMPeLU/9NBDMsZo1apVXu0xMTG68MILrefdunWT0+nUd999d873CQsL05AhQ6y2Ro0a6YEHHlB+fr42btxY6WNwu92Kjo62nrdp00a33HKL1qxZo6KiIq/aMWPGlOqXJE2YMMGrvWT2Jzk52as9KipKbrfbet6rVy9Jv52ebNOmTan2ssbl1GXEJafACgsL9dFHH53jSMsnIyNDe/bs0Z133qlffvlFP//8s37++WcdOXJEffv21aZNm0r9XJ8+LldeeaV++eUXeTweSbJO8ZTMMJW4//77K9y/8vwMBQcHa8uWLTpw4ECF94/6hwCDeiM/P98rLJzuz3/+sy6//HKNGjVKoaGhGjx4sJYtW1ahMHPBBRdU6ILdDh06eD338fFR+/btz3r9R1X4/vvvFR4eXmo8OnfubG0/1al/pEs0bdpUv/766znfp0OHDl6nU872PhVx+thJ0h//+EcdPXpUP/30k1d7ZGRkqX75+vqqffv2Xu1hYWEKDg4+5/G7XC5JUkRERJntp4+Lr6+v/vCHP5Tqq6Qq+17v2bNHkhQfH6+WLVt6PRYtWqSCggLl5eV5veb042ratKlX/0vG6fTxO33cyqM8P0OzZs3Sjh07FBERocsuu0zTpk07Z0hG/cU1MKgX/vvf/yovL++s//AGBARo06ZNWr9+vZKTk7V69WotXbpU1157rdauXasGDRqc830qct1KeZ3pZntFRUXl6lNVONP7mNMu+K2tzvR9Ke+NDM90/LVpXEqC9uzZs8+4xD8oKMjr+fnsf3ne64477tCVV16pd999V2vXrtXs2bP197//Xe+884769+9f5X2CvTEDg3rhX//6lyQpNjb2rHW+vr7q27evnnvuOX399dd66qmnlJqaqvXr10sq/x+88ir5X3MJY4z27t3rtWKoadOmZd4c7PRZgor0rW3btjpw4ECpU2q7du2ytleFtm3bas+ePaVmsarifU4fO0n69ttv1bhx43NeANy2bVsVFxeX2kdOTo5yc3Or7PhLFBcXl5pJ+PbbbyXJ+l7/3p+tktMzTqdTMTExZT4qupy+ZJwyMzO92vfu3Vuqtqp+N1q1aqX77rtPK1asUGZmppo3b66nnnqqSvaNuoUAgzovNTVVTz75pCIjIzV06NAz1h06dKhUW8n/ZEuW1pbcP6Sq7jb6f//3f14h4u2339aPP/7o9b/NCy+8UJ9++qkKCwuttpUrV5Zabl2Rvt1www0qKirSiy++6NU+Z84c+fj4VNn/dm+44QZlZ2dr6dKlVtvJkyf1wgsvKCgoSFdffXWl952Wlua1vHb//v1677331K9fv3POTN1www2SpLlz53q1P/fcc5JUapVNVTh1rI0xevHFF9WoUSP17dtXktS4cWNJlf/Zio6O1oUXXqhnnnlG+fn5pbafflqtPEoC//z5873aX3jhhVK1v/d3o6ioqNQprpCQEIWHh5da2g5InEJCHbNq1Srt2rVLJ0+eVE5OjlJTU5WSkqK2bdvq/fffl7+//xlfO336dG3atElxcXFq27atDh48qPnz56t169bWvUMuvPBCBQcHa+HChWrSpIkCAwPVq1evUtcIlFezZs10xRVXaMSIEcrJydHcuXPVvn17r6Xeo0aN0ttvv63rr79ed9xxh/7zn//otdde87ogsqJ9u+mmm9SnTx89+uij2rdvn7p37661a9fqvffeU2JiYql9V9bo0aP18ssva/jw4UpPT1e7du309ttv65NPPtHcuXPPek3SuXTp0kWxsbFey6glWXezPZvu3bsrPj5er7zyinJzc3X11Vdr69atWrJkiW699Vb16dOn0v0qi7+/v1avXq34+Hj16tVLq1atUnJysh555BFrtiggIEBRUVFaunSp/vjHP6pZs2bq0qVLqSXhZ+Lr66tFixapf//+uuiiizRixAhdcMEF+uGHH7R+/Xo5nU598MEHFep3dHS0Bg4cqLlz5+qXX36xllGXzB6dOutSckH1o48+qsGDB6tRo0a66aabyn3TwMOHD6t169YaNGiQunfvrqCgIH300Ufatm2bnn322Qr1G/VEDa6AAqpMyTLqkoefn58JCwsz1113nZk3b57Xct0Spy+jXrdunbnllltMeHi48fPzM+Hh4WbIkCHm22+/9Xrde++9Z6KiokzDhg29li1fffXV5qKLLiqzf2daRv3mm2+aKVOmmJCQEBMQEGDi4uK8luWWePbZZ80FF1xgHA6Hufzyy81nn31Wap9n69vpy6iNMebw4cNm/PjxJjw83DRq1Mh06NDBzJ4922sJrjG/LYEta2nrmZZ3ny4nJ8eMGDHCtGjRwvj5+ZmuXbuWudS7osuoExISzGuvvWY6dOhgHA6HueSSS7yWLxvzv+9xWUvjT5w4YZ544gkTGRlpGjVqZCIiIsyUKVPM8ePHy9WvssYlMzPTSDKzZ8+22uLj401gYKD5z3/+Y/r162caN25sQkNDzeOPP26Kioq8Xr9582YTHR1t/Pz8zrmk+vRl1CW++OILM2DAANO8eXPjcDhM27ZtzR133GHWrVt3znEp+T06dSn0kSNHTEJCgmnWrJkJCgoyt956q9m9e7eRZGbOnOn1+ieffNJccMEFxtfX12s/5fkZKigoMBMnTjTdu3c3TZo0MYGBgaZ79+5m/vz5ZxwD1G8+xtjkKjwA+P/5+PgoISGh1Cmw2mj48OF6++23yzytY1cZGRm65JJL9Nprr531tCxQnbgGBgBwRseOHSvVNnfuXPn6+uqqq66qgR4Bv+EaGADAGc2aNUvp6enq06ePGjZsqFWrVmnVqlUaPXp0qfvgAOcTAQYAcEZ/+tOflJKSoieffFL5+flq06aNpk2bpkcffbSmu4Z6jmtgAACA7XANDAAAsB0CDAAAsJ06ew1McXGxDhw4oCZNmlT57d8BAED1MMbo8OHDCg8PL/VBsKeqswHmwIEDXCEPAIBN7d+/X61btz7j9jobYEpuUb5//345nc4a7k09deSIFB7+29cHDkjlvKU4AKD+8ng8ioiIOOdHjdTZAFNy2sjpdBJgasqpH6jndBJgAADldq7LP7iIFwAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2E7Dmu5AfdJucrLX830z42qoJwAA2BszMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYqFGDatWsnHx+fUo+EhARJ0vHjx5WQkKDmzZsrKChIAwcOVE5Ojtc+srKyFBcXp8aNGyskJEQTJ07UyZMnvWo2bNigHj16yOFwqH379kpKSvp9RwkAAOqUCgWYbdu26ccff7QeKSkpkqTbb79dkjR+/Hh98MEHWr58uTZu3KgDBw5owIAB1uuLiooUFxenwsJCbd68WUuWLFFSUpKmTp1q1WRmZiouLk59+vRRRkaGEhMTNWrUKK1Zs6YqjhcAANQBPsYYU9kXJyYmauXKldqzZ488Ho9atmypN954Q4MGDZIk7dq1S507d1ZaWpp69+6tVatW6cYbb9SBAwcUGhoqSVq4cKEmTZqkn376SX5+fpo0aZKSk5O1Y8cO630GDx6s3NxcrV69utx983g8crlcysvLk9PprOwhVql2k5O9nu+bGVdDPTlPjhyRgoJ++zo/XwoMrNn+AABqvfL+/a70NTCFhYV67bXXdM8998jHx0fp6ek6ceKEYmJirJpOnTqpTZs2SktLkySlpaWpa9euVniRpNjYWHk8Hu3cudOqOXUfJTUl+ziTgoICeTwerwcAAKibKh1gVqxYodzcXA0fPlySlJ2dLT8/PwUHB3vVhYaGKjs726o5NbyUbC/ZdrYaj8ejY8eOnbE/M2bMkMvlsh4RERGVPTQAAFDLVTrA/OMf/1D//v0VHh5elf2ptClTpigvL8967N+/v6a7BAAAqknDyrzo+++/10cffaR33nnHagsLC1NhYaFyc3O9ZmFycnIUFhZm1WzdutVrXyWrlE6tOX3lUk5OjpxOpwICAs7YJ4fDIYfDUZnDAQAANlOpGZjFixcrJCREcXH/uwg1OjpajRo10rp166y23bt3KysrS263W5Lkdru1fft2HTx40KpJSUmR0+lUVFSUVXPqPkpqSvYBAABQ4QBTXFysxYsXKz4+Xg0b/m8Cx+VyaeTIkZowYYLWr1+v9PR0jRgxQm63W71795Yk9evXT1FRURo2bJi+/PJLrVmzRo899pgSEhKs2ZMxY8bou+++08MPP6xdu3Zp/vz5WrZsmcaPH19FhwwAAOyuwqeQPvroI2VlZemee+4ptW3OnDny9fXVwIEDVVBQoNjYWM2fP9/a3qBBA61cuVJjx46V2+1WYGCg4uPjNX36dKsmMjJSycnJGj9+vObNm6fWrVtr0aJFio2NreQhAgCAuuZ33QemNuM+MLUA94EBAFRQtd8HBgAAoKYQYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO0QYAAAgO1UOMD88MMPuuuuu9S8eXMFBASoa9eu+uyzz6ztxhhNnTpVrVq1UkBAgGJiYrRnzx6vfRw6dEhDhw6V0+lUcHCwRo4cqfz8fK+ar776SldeeaX8/f0VERGhWbNmVfIQa0a7ycmlHgAAoGpUKMD8+uuvuvzyy9WoUSOtWrVKX3/9tZ599lk1bdrUqpk1a5aef/55LVy4UFu2bFFgYKBiY2N1/Phxq2bo0KHauXOnUlJStHLlSm3atEmjR4+2tns8HvXr109t27ZVenq6Zs+erWnTpumVV16pgkMGAAB252OMMeUtnjx5sj755BP9+9//LnO7MUbh4eF66KGH9Je//EWSlJeXp9DQUCUlJWnw4MH65ptvFBUVpW3btqlnz56SpNWrV+uGG27Qf//7X4WHh2vBggV69NFHlZ2dLT8/P+u9V6xYoV27dpWrrx6PRy6XS3l5eXI6neU9xCpTnhmXfTPjzkNPatCRI1JQ0G9f5+dLgYE12x8AQK1X3r/fFZqBef/999WzZ0/dfvvtCgkJ0SWXXKJXX33V2p6Zmans7GzFxMRYbS6XS7169VJaWpokKS0tTcHBwVZ4kaSYmBj5+vpqy5YtVs1VV11lhRdJio2N1e7du/Xrr7+W2beCggJ5PB6vBwAAqJsqFGC+++47LViwQB06dNCaNWs0duxYPfDAA1qyZIkkKTs7W5IUGhrq9brQ0FBrW3Z2tkJCQry2N2zYUM2aNfOqKWsfp77H6WbMmCGXy2U9IiIiKnJoAADARioUYIqLi9WjRw89/fTTuuSSSzR69Gjde++9WrhwYXX1r9ymTJmivLw867F///6a7hIAAKgmFQowrVq1UlRUlFdb586dlZWVJUkKCwuTJOXk5HjV5OTkWNvCwsJ08OBBr+0nT57UoUOHvGrK2sep73E6h8Mhp9Pp9QAAAHVThQLM5Zdfrt27d3u1ffvtt2rbtq0kKTIyUmFhYVq3bp213ePxaMuWLXK73ZIkt9ut3NxcpaenWzWpqakqLi5Wr169rJpNmzbpxIkTVk1KSoo6duzoteIJAADUTxUKMOPHj9enn36qp59+Wnv37tUbb7yhV155RQkJCZIkHx8fJSYm6m9/+5vef/99bd++XXfffbfCw8N16623Svptxub666/Xvffeq61bt+qTTz7RuHHjNHjwYIWHh0uS7rzzTvn5+WnkyJHauXOnli5dqnnz5mnChAlVe/QAAMCWGlak+NJLL9W7776rKVOmaPr06YqMjNTcuXM1dOhQq+bhhx/WkSNHNHr0aOXm5uqKK67Q6tWr5e/vb9W8/vrrGjdunPr27StfX18NHDhQzz//vLXd5XJp7dq1SkhIUHR0tFq0aKGpU6d63SsGAADUXxW6D4ydcB+YWoD7wAAAKqha7gMDAABQGxBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7RBgAACA7TSs6Q7UZ+0mJ5dq2zczrgZ6AgCAvTADAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbIcAAwAAbKdCAWbatGny8fHxenTq1Mnafvz4cSUkJKh58+YKCgrSwIEDlZOT47WPrKwsxcXFqXHjxgoJCdHEiRN18uRJr5oNGzaoR48ecjgcat++vZKSkip/hAAAoM6p8AzMRRddpB9//NF6fPzxx9a28ePH64MPPtDy5cu1ceNGHThwQAMGDLC2FxUVKS4uToWFhdq8ebOWLFmipKQkTZ061arJzMxUXFyc+vTpo4yMDCUmJmrUqFFas2bN7zxUAABQVzSs8AsaNlRYWFip9ry8PP3jH//QG2+8oWuvvVaStHjxYnXu3FmffvqpevfurbVr1+rrr7/WRx99pNDQUF188cV68sknNWnSJE2bNk1+fn5auHChIiMj9eyzz0qSOnfurI8//lhz5sxRbGzs7zxcAABQF1R4BmbPnj0KDw/XH/7wBw0dOlRZWVmSpPT0dJ04cUIxMTFWbadOndSmTRulpaVJktLS0tS1a1eFhoZaNbGxsfJ4PNq5c6dVc+o+SmpK9nEmBQUF8ng8Xg8AAFA3VSjA9OrVS0lJSVq9erUWLFigzMxMXXnllTp8+LCys7Pl5+en4OBgr9eEhoYqOztbkpSdne0VXkq2l2w7W43H49GxY8fO2LcZM2bI5XJZj4iIiIocGgAAsJEKnULq37+/9XW3bt3Uq1cvtW3bVsuWLVNAQECVd64ipkyZogkTJljPPR4PIQYAgDrqdy2jDg4O1h//+Eft3btXYWFhKiwsVG5urldNTk6Odc1MWFhYqVVJJc/PVeN0Os8akhwOh5xOp9cDAADUTb8rwOTn5+s///mPWrVqpejoaDVq1Ejr1q2ztu/evVtZWVlyu92SJLfbre3bt+vgwYNWTUpKipxOp6KioqyaU/dRUlOyDwAAgAoFmL/85S/auHGj9u3bp82bN+u2225TgwYNNGTIELlcLo0cOVITJkzQ+vXrlZ6erhEjRsjtdqt3796SpH79+ikqKkrDhg3Tl19+qTVr1uixxx5TQkKCHA6HJGnMmDH67rvv9PDDD2vXrl2aP3++li1bpvHjx1f90QMAAFuq0DUw//3vfzVkyBD98ssvatmypa644gp9+umnatmypSRpzpw58vX11cCBA1VQUKDY2FjNnz/fen2DBg20cuVKjR07Vm63W4GBgYqPj9f06dOtmsjISCUnJ2v8+PGaN2+eWrdurUWLFrGEGgAAWHyMMaamO1EdPB6PXC6X8vLyauR6mHaTkyv1un0z46q4JzXoyBEpKOi3r/PzpcDAmu0PAKDWK+/fbz4LCQAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2A4BBgAA2E7Dmu5AXdFucnJNdwEAgHqDGRgAAGA7BBgAAGA7BBgAAGA7BBgAAGA7BBgAAGA7BBgAAGA7vyvAzJw5Uz4+PkpMTLTajh8/roSEBDVv3lxBQUEaOHCgcnJyvF6XlZWluLg4NW7cWCEhIZo4caJOnjzpVbNhwwb16NFDDodD7du3V1JS0u/pKgAAqEMqHWC2bduml19+Wd26dfNqHz9+vD744AMtX75cGzdu1IEDBzRgwABre1FRkeLi4lRYWKjNmzdryZIlSkpK0tSpU62azMxMxcXFqU+fPsrIyFBiYqJGjRqlNWvWVLa7AACgDqlUgMnPz9fQoUP16quvqmnTplZ7Xl6e/vGPf+i5557Ttddeq+joaC1evFibN2/Wp59+Kklau3atvv76a7322mu6+OKL1b9/fz355JN66aWXVFhYKElauHChIiMj9eyzz6pz584aN26cBg0apDlz5lTBIQMAALurVIBJSEhQXFycYmJivNrT09N14sQJr/ZOnTqpTZs2SktLkySlpaWpa9euCg0NtWpiY2Pl8Xi0c+dOq+b0fcfGxlr7KEtBQYE8Ho/XAwAA1E0V/iiBt956S59//rm2bdtWalt2drb8/PwUHBzs1R4aGqrs7Gyr5tTwUrK9ZNvZajwej44dO6aAgIBS7z1jxgw98cQTFT0cAABgQxWagdm/f78efPBBvf766/L396+uPlXKlClTlJeXZz32799f010CAADVpEIzMOnp6Tp48KB69OhhtRUVFWnTpk168cUXtWbNGhUWFio3N9drFiYnJ0dhYWGSpLCwMG3dutVrvyWrlE6tOX3lUk5OjpxOZ5mzL5LkcDjkcDgqcji10ukfCrlvZlwN9QQAgNqrQgGmb9++2r59u1fbiBEj1KlTJ02aNEkRERFq1KiR1q1bp4EDB0qSdu/eraysLLndbkmS2+3WU089pYMHDyokJESSlJKSIqfTqaioKKvmww8/9HqflJQUax/1SVmfck2oAQDUdxUKME2aNFGXLl282gIDA9W8eXOrfeTIkZowYYKaNWsmp9Op+++/X263W71795Yk9evXT1FRURo2bJhmzZql7OxsPfbYY0pISLBmUMaMGaMXX3xRDz/8sO655x6lpqZq2bJlSk4u/cccAADUPxW+iPdc5syZI19fXw0cOFAFBQWKjY3V/Pnzre0NGjTQypUrNXbsWLndbgUGBio+Pl7Tp0+3aiIjI5WcnKzx48dr3rx5at26tRYtWqTY2Niq7i4AALAhH2OMqelOVAePxyOXy6W8vDw5nc5qf7+yTvVUF9ucQjpyRAoK+u3r/HwpMLBm+wMAqPXK+/ebz0ICAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC2Q4ABAAC207CmO4CKazc52ev5vplxNdQTAABqBjMwAADAdggwAADAdggwAADAdggwAADAdriItw44/aJeiQt7AQB1GzMwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdggwAADAdioUYBYsWKBu3brJ6XTK6XTK7XZr1apV1vbjx48rISFBzZs3V1BQkAYOHKicnByvfWRlZSkuLk6NGzdWSEiIJk6cqJMnT3rVbNiwQT169JDD4VD79u2VlJRU+SMEAAB1ToUCTOvWrTVz5kylp6frs88+07XXXqtbbrlFO3fulCSNHz9eH3zwgZYvX66NGzfqwIEDGjBggPX6oqIixcXFqbCwUJs3b9aSJUuUlJSkqVOnWjWZmZmKi4tTnz59lJGRocTERI0aNUpr1qypokMGAAB252OMMb9nB82aNdPs2bM1aNAgtWzZUm+88YYGDRokSdq1a5c6d+6stLQ09e7dW6tWrdKNN96oAwcOKDQ0VJK0cOFCTZo0ST/99JP8/Pw0adIkJScna8eOHdZ7DB48WLm5uVq9enW5++XxeORyuZSXlyen0/l7DrFc2k1Orvb3qIh9M+NqugvSkSNSUNBvX+fnS4GBNdsfAECtV96/35W+BqaoqEhvvfWWjhw5IrfbrfT0dJ04cUIxMTFWTadOndSmTRulpaVJktLS0tS1a1crvEhSbGysPB6PNYuTlpbmtY+SmpJ9nElBQYE8Ho/XAwAA1E0VDjDbt29XUFCQHA6HxowZo3fffVdRUVHKzs6Wn5+fgoODvepDQ0OVnZ0tScrOzvYKLyXbS7adrcbj8ejYsWNn7NeMGTPkcrmsR0REREUPDQAA2ESFA0zHjh2VkZGhLVu2aOzYsYqPj9fXX39dHX2rkClTpigvL8967N+/v6a7BAAAqknDir7Az89P7du3lyRFR0dr27Ztmjdvnv785z+rsLBQubm5XrMwOTk5CgsLkySFhYVp69atXvsrWaV0as3pK5dycnLkdDoVEBBwxn45HA45HI6KHg4AALCh330fmOLiYhUUFCg6OlqNGjXSunXrrG27d+9WVlaW3G63JMntdmv79u06ePCgVZOSkiKn06moqCir5tR9lNSU7AMAAKBCMzBTpkxR//791aZNGx0+fFhvvPGGNmzYoDVr1sjlcmnkyJGaMGGCmjVrJqfTqfvvv19ut1u9e/eWJPXr109RUVEaNmyYZs2apezsbD322GNKSEiwZk/GjBmjF198UQ8//LDuuecepaamatmyZUpOrl2rfAAAQM2pUIA5ePCg7r77bv34449yuVzq1q2b1qxZo+uuu06SNGfOHPn6+mrgwIEqKChQbGys5s+fb72+QYMGWrlypcaOHSu3263AwEDFx8dr+vTpVk1kZKSSk5M1fvx4zZs3T61bt9aiRYsUGxtbRYcMAADs7nffB6a24j4w3AcGAGA/1X4fGAAAgJpS4VVIsIfTZ4RqxYwMAABVhBkYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOwQYAABgOw1rugM4P9pNTi7Vtm9mXA30BACA348ZGAAAYDsEGAAAYDsEGAAAYDsVCjAzZszQpZdeqiZNmigkJES33nqrdu/e7VVz/PhxJSQkqHnz5goKCtLAgQOVk5PjVZOVlaW4uDg1btxYISEhmjhxok6ePOlVs2HDBvXo0UMOh0Pt27dXUlJS5Y4QAADUORUKMBs3blRCQoI+/fRTpaSk6MSJE+rXr5+OHDli1YwfP14ffPCBli9fro0bN+rAgQMaMGCAtb2oqEhxcXEqLCzU5s2btWTJEiUlJWnq1KlWTWZmpuLi4tSnTx9lZGQoMTFRo0aN0po1a6rgkAEAgN35GGNMZV/8008/KSQkRBs3btRVV12lvLw8tWzZUm+88YYGDRokSdq1a5c6d+6stLQ09e7dW6tWrdKNN96oAwcOKDQ0VJK0cOFCTZo0ST/99JP8/Pw0adIkJScna8eOHdZ7DR48WLm5uVq9enW5+ubxeORyuZSXlyen01nZQyy3slb51HbVvgrpyBEpKOi3r/PzpcDA6n0/AIDtlffv9++6BiYvL0+S1KxZM0lSenq6Tpw4oZiYGKumU6dOatOmjdLS0iRJaWlp6tq1qxVeJCk2NlYej0c7d+60ak7dR0lNyT7KUlBQII/H4/UAAAB1U6UDTHFxsRITE3X55ZerS5cukqTs7Gz5+fkpODjYqzY0NFTZ2dlWzanhpWR7ybaz1Xg8Hh07dqzM/syYMUMul8t6REREVPbQAABALVfpAJOQkKAdO3borbfeqsr+VNqUKVOUl5dnPfbv31/TXQIAANWkUnfiHTdunFauXKlNmzapdevWVntYWJgKCwuVm5vrNQuTk5OjsLAwq2br1q1e+ytZpXRqzekrl3JycuR0OhUQEFBmnxwOhxwOR2UOBwAA2EyFZmCMMRo3bpzeffddpaamKjIy0mt7dHS0GjVqpHXr1lltu3fvVlZWltxutyTJ7XZr+/btOnjwoFWTkpIip9OpqKgoq+bUfZTUlOwDAADUbxWagUlISNAbb7yh9957T02aNLGuWXG5XAoICJDL5dLIkSM1YcIENWvWTE6nU/fff7/cbrd69+4tSerXr5+ioqI0bNgwzZo1S9nZ2XrssceUkJBgzaCMGTNGL774oh5++GHdc889Sk1N1bJly5ScbL+VPgAAoOpVaAZmwYIFysvL0zXXXKNWrVpZj6VLl1o1c+bM0Y033qiBAwfqqquuUlhYmN555x1re4MGDbRy5Uo1aNBAbrdbd911l+6++25Nnz7dqomMjFRycrJSUlLUvXt3Pfvss1q0aJFiY2Or4JABAIDd/a77wNRm3Afm3LgPDACgtjkv94EBAACoCQQYAABgO5VaRo26oazTXtV+WgkAgCrADAwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALAdAgwAALCdhjXdAdQu7SYnez3fNzOuhnoCAMCZMQMDAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABshwADAABsp8IBZtOmTbrpppsUHh4uHx8frVixwmu7MUZTp05Vq1atFBAQoJiYGO3Zs8er5tChQxo6dKicTqeCg4M1cuRI5efne9V89dVXuvLKK+Xv76+IiAjNmjWr4kcHAADqpAoHmCNHjqh79+566aWXytw+a9YsPf/881q4cKG2bNmiwMBAxcbG6vjx41bN0KFDtXPnTqWkpGjlypXatGmTRo8ebW33eDzq16+f2rZtq/T0dM2ePVvTpk3TK6+8UolDBAAAdU2FP426f//+6t+/f5nbjDGaO3euHnvsMd1yyy2SpP/7v/9TaGioVqxYocGDB+ubb77R6tWrtW3bNvXs2VOS9MILL+iGG27QM888o/DwcL3++usqLCzUP//5T/n5+emiiy5SRkaGnnvuOa+gAwAA6qcqvQYmMzNT2dnZiomJsdpcLpd69eqltLQ0SVJaWpqCg4Ot8CJJMTEx8vX11ZYtW6yaq666Sn5+flZNbGysdu/erV9//bXM9y4oKJDH4/F6AACAuqlKA0x2drYkKTQ01Ks9NDTU2padna2QkBCv7Q0bNlSzZs28asrax6nvcboZM2bI5XJZj4iIiN9/QAAAoFaqM6uQpkyZory8POuxf//+mu4SAACoJhW+BuZswsLCJEk5OTlq1aqV1Z6Tk6OLL77Yqjl48KDX606ePKlDhw5Zrw8LC1NOTo5XTcnzkprTORwOORyOKjkO/E+7ycml2vbNjKuBngAA8D9VGmAiIyMVFhamdevWWYHF4/Foy5YtGjt2rCTJ7XYrNzdX6enpio6OliSlpqaquLhYvXr1smoeffRRnThxQo0aNZIkpaSkqGPHjmratGlVdrlSyvqjDgAAzp8Kn0LKz89XRkaGMjIyJP124W5GRoaysrLk4+OjxMRE/e1vf9P777+v7du36+6771Z4eLhuvfVWSVLnzp11/fXX695779XWrVv1ySefaNy4cRo8eLDCw8MlSXfeeaf8/Pw0cuRI7dy5U0uXLtW8efM0YcKEKjtwAABgXxWegfnss8/Up08f63lJqIiPj1dSUpIefvhhHTlyRKNHj1Zubq6uuOIKrV69Wv7+/tZrXn/9dY0bN059+/aVr6+vBg4cqOeff97a7nK5tHbtWiUkJCg6OlotWrTQ1KlTWUINAAAkST7GGFPTnagOHo9HLpdLeXl5cjqdVbrv+n4KqdzXwBw5IgUF/fZ1fr4UGFh9nQIA1Anl/ftdZ1YhAQCA+qNKL+JF/XD6DBSrkgAA5xszMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHYIMAAAwHb4MEf8bqd/uKPEBzwCAKoXMzAAAMB2CDAAAMB2CDAAAMB2CDAAAMB2uIgX1aLd5GQFFB7XNzXdEQBAncQMDAAAsB0CDAAAsB0CDAAAsB0CDAAAsB0u4sV50fmvq3XMz9+rjbv1AgAqixkYAABgOwQYAABgO5xCQo05/UMgOaUEACgvZmAAAIDtMAODWuP0GRmJWRkAQNkIMKjVOM0EACgLp5AAAIDtMAMDWynrNNPpmKUBgLqPGRgAAGA7zMCgzmGWBgDqvlo9A/PSSy+pXbt28vf3V69evbR169aa7hIAAKgFau0MzNKlSzVhwgQtXLhQvXr10ty5cxUbG6vdu3crJCSkprsHmyvPLE1ZmLkBgNqh1gaY5557Tvfee69GjBghSVq4cKGSk5P1z3/+U5MnT67h3qG+4vQUANQOtTLAFBYWKj09XVOmTLHafH19FRMTo7S0tDJfU1BQoIKCAut5Xl6eJMnj8VR5/4oLjlb5PuuiosLjKhn9ooKjKjbFNdqf86XN+OVVsp8dT8Ses6bL42vO+Zry1ABAbVHyd9sYc9a6Whlgfv75ZxUVFSk0NNSrPTQ0VLt27SrzNTNmzNATTzxRqj0iIqJa+ojycZV8Mf/umuyGLbnmVs9rKrNfADjfDh8+LJfLdcbttTLAVMaUKVM0YcIE63lxcbEOHTqk5s2by8fHx6vW4/EoIiJC+/fvl9PpPN9drdUYm7NjfM6MsTk7xufMGJuzq2/jY4zR4cOHFR4efta6WhlgWrRooQYNGignJ8erPScnR2FhYWW+xuFwyOFweLUFBwef9X2cTme9+GGoDMbm7BifM2Nszo7xOTPG5uzq0/icbealRK1cRu3n56fo6GitW7fOaisuLta6devkdrtrsGcAAKA2qJUzMJI0YcIExcfHq2fPnrrssss0d+5cHTlyxFqVBAAA6q9aG2D+/Oc/66efftLUqVOVnZ2tiy++WKtXry51YW9lOBwOPf7446VOOYGxORfG58wYm7NjfM6MsTk7xqdsPuZc65QAAABqmVp5DQwAAMDZEGAAAIDtEGAAAIDtEGAAAIDtEGAAAIDt1MsA89JLL6ldu3by9/dXr169tHXr1pruUrWaMWOGLr30UjVp0kQhISG69dZbtXv3bq+a48ePKyEhQc2bN1dQUJAGDhxY6k7IWVlZiouLU+PGjRUSEqKJEyfq5MmT5/NQzouZM2fKx8dHiYmJVlt9Hp8ffvhBd911l5o3b66AgAB17dpVn332mbXdGKOpU6eqVatWCggIUExMjPbs2eO1j0OHDmno0KFyOp0KDg7WyJEjlZ+ff74PpcoVFRXpr3/9qyIjIxUQEKALL7xQTz75pNeH0NWX8dm0aZNuuukmhYeHy8fHRytWrPDaXlXj8NVXX+nKK6+Uv7+/IiIiNGvWrOo+tCpxtvE5ceKEJk2apK5duyowMFDh4eG6++67deDAAa991OXxqRRTz7z11lvGz8/P/POf/zQ7d+409957rwkODjY5OTk13bVqExsbaxYvXmx27NhhMjIyzA033GDatGlj8vPzrZoxY8aYiIgIs27dOvPZZ5+Z3r17mz/96U/W9pMnT5ouXbqYmJgY88UXX5gPP/zQtGjRwkyZMqUmDqnabN261bRr185069bNPPjgg1Z7fR2fQ4cOmbZt25rhw4ebLVu2mO+++86sWbPG7N2716qZOXOmcblcZsWKFebLL780N998s4mMjDTHjh2zaq6//nrTvXt38+mnn5p///vfpn379mbIkCE1cUhV6qmnnjLNmzc3K1euNJmZmWb58uUmKCjIzJs3z6qpL+Pz4YcfmkcffdS88847RpJ59913vbZXxTjk5eWZ0NBQM3ToULNjxw7z5ptvmoCAAPPyyy+fr8OstLONT25uromJiTFLly41u3btMmlpaeayyy4z0dHRXvuoy+NTGfUuwFx22WUmISHBel5UVGTCw8PNjBkzarBX59fBgweNJLNx40ZjzG+/PI0aNTLLly+3ar755hsjyaSlpRljfvvl8/X1NdnZ2VbNggULjNPpNAUFBef3AKrJ4cOHTYcOHUxKSoq5+uqrrQBTn8dn0qRJ5oorrjjj9uLiYhMWFmZmz55tteXm5hqHw2HefPNNY4wxX3/9tZFktm3bZtWsWrXK+Pj4mB9++KH6On8exMXFmXvuucerbcCAAWbo0KHGmPo7Pqf/ga6qcZg/f75p2rSp1+/UpEmTTMeOHav5iKpWWQHvdFu3bjWSzPfff2+MqV/jU1716hRSYWGh0tPTFRMTY7X5+voqJiZGaWlpNdiz8ysvL0+S1KxZM0lSenq6Tpw44TUunTp1Ups2baxxSUtLU9euXb3uhBwbGyuPx6OdO3eex95Xn4SEBMXFxXmNg1S/x+f9999Xz549dfvttyskJESXXHKJXn31VWt7ZmamsrOzvcbG5XKpV69eXmMTHBysnj17WjUxMTHy9fXVli1bzt/BVIM//elPWrdunb799ltJ0pdffqmPP/5Y/fv3l8T4lKiqcUhLS9NVV10lPz8/qyY2Nla7d+/Wr7/+ep6O5vzIy8uTj4+P9aHEjE9ptfajBKrDzz//rKKiolIfRxAaGqpdu3bVUK/Or+LiYiUmJuryyy9Xly5dJEnZ2dny8/Mr9endoaGhys7OtmrKGreSbXb31ltv6fPPP9e2bdtKbavP4/Pdd99pwYIFmjBhgh555BFt27ZNDzzwgPz8/BQfH28dW1nHfurYhISEeG1v2LChmjVrZuuxkaTJkyfL4/GoU6dOatCggYqKivTUU09p6NChklTvx6dEVY1Ddna2IiMjS+2jZFvTpk2rpf/n2/HjxzVp0iQNGTLE+vRpxqe0ehVg8Nssw44dO/Txxx/XdFdqjf379+vBBx9USkqK/P39a7o7tUpxcbF69uypp59+WpJ0ySWXaMeOHVq4cKHi4+NruHc1b9myZXr99df1xhtv6KKLLlJGRoYSExMVHh7O+KBSTpw4oTvuuEPGGC1YsKCmu1Or1atTSC1atFCDBg1KrR7JyclRWFhYDfXq/Bk3bpxWrlyp9evXq3Xr1lZ7WFiYCgsLlZub61V/6riEhYWVOW4l2+wsPT1dBw8eVI8ePdSwYUM1bNhQGzdu1PPPP6+GDRsqNDS03o5Pq1atFBUV5dXWuXNnZWVlSfrfsZ3tdyosLEwHDx702n7y5EkdOnTI1mMjSRMnTtTkyZM1ePBgde3aVcOGDdP48eM1Y8YMSYxPiaoah7r6e1aiJLx8//33SklJsWZfJManLPUqwPj5+Sk6Olrr1q2z2oqLi7Vu3Tq53e4a7Fn1MsZo3Lhxevfdd5WamlpqijE6OlqNGjXyGpfdu3crKyvLGhe3263t27d7/QKV/IKd/gfObvr27avt27crIyPDevTs2VNDhw61vq6v43P55ZeXWnL/7bffqm3btpKkyMhIhYWFeY2Nx+PRli1bvMYmNzdX6enpVk1qaqqKi4vVq1ev83AU1efo0aPy9fX+Z7RBgwYqLi6WxPiUqKpxcLvd2rRpk06cOGHVpKSkqGPHjrY/PVISXvbs2aOPPvpIzZs399pe38enTDV9FfH59tZbbxmHw2GSkpLM119/bUaPHm2Cg4O9Vo/UNWPHjjUul8ts2LDB/Pjjj9bj6NGjVs2YMWNMmzZtTGpqqvnss8+M2+02brfb2l6yTLhfv34mIyPDrF692rRs2dL2y4TP5NRVSMbU3/HZunWradiwoXnqqafMnj17zOuvv24aN25sXnvtNatm5syZJjg42Lz33nvmq6++MrfcckuZy2MvueQSs2XLFvPxxx+bDh062G6ZcFni4+PNBRdcYC2jfuedd0yLFi3Mww8/bNXUl/E5fPiw+eKLL8wXX3xhJJnnnnvOfPHFF9YqmqoYh9zcXBMaGmqGDRtmduzYYd566y3TuHFjWywTPtv4FBYWmptvvtm0bt3aZGRkeP07feqKoro8PpVR7wKMMca88MILpk2bNsbPz89cdtll5tNPP63pLlUrSWU+Fi9ebNUcO3bM3HfffaZp06amcePG5rbbbjM//vij13727dtn+vfvbwICAkyLFi3MQw89ZE6cOHGej+b8OD3A1Ofx+eCDD0yXLl2Mw+EwnTp1Mq+88orX9uLiYvPXv/7VhIaGGofDYfr27Wt2797tVfPLL7+YIUOGmKCgION0Os2IESPM4cOHz+dhVAuPx2MefPBB06ZNG+Pv72/+8Ic/mEcffdTrj059GZ/169eX+e9MfHy8MabqxuHLL780V1xxhXE4HOaCCy4wM2fOPF+H+LucbXwyMzPP+O/0+vXrrX3U5fGpDB9jTrllJAAAgA3Uq2tgAABA3UCAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtkOAAQAAtvP/AUEYNAunFIVYAAAAAElFTkSuQmCC",
80
+ "text/plain": [
81
+ "<Figure size 640x480 with 1 Axes>"
82
+ ]
83
+ },
84
+ "metadata": {},
85
+ "output_type": "display_data"
86
+ }
87
+ ],
88
+ "source": [
89
+ "import matplotlib.pyplot as plt\n",
90
+ "\n",
91
+ "lens = [len(x[\"prompt\"][\"input_ids\"]) for x in data[\"train\"]]\n",
92
+ "plt.hist(lens, bins=100)\n",
93
+ "plt.title(\"Distribution of prompt lengths\")\n",
94
+ "plt.axvline(256, color=\"red\")"
95
+ ]
96
+ },
97
+ {
98
+ "cell_type": "code",
99
+ "execution_count": 3,
100
+ "metadata": {},
101
+ "outputs": [
102
+ {
103
+ "data": {
104
+ "text/plain": [
105
+ "<matplotlib.lines.Line2D at 0x7f6eef316ce0>"
106
+ ]
107
+ },
108
+ "execution_count": 3,
109
+ "metadata": {},
110
+ "output_type": "execute_result"
111
+ },
112
+ {
113
+ "data": {
114
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkUAAAGzCAYAAAAhXWNYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABagUlEQVR4nO3dd3xT9f4/8FeSNulMB12UllVG2WhZdeGoVOhVUVRElMpQ8RZZ9wLiAHHB1auCyhD5fcWLKMPNEMQC4iirUNnIKJu2lNKkLZ3J+/dHyaGhBTrShiSv5+ORB805n5zzOZ/kJC/O+XzOUYmIgIiIiMjFqe1dASIiIqIbAUMRERERERiKiIiIiAAwFBEREREBYCgiIiIiAsBQRERERASAoYiIiIgIAEMREREREQCGIiIiIiIADEUNbuPGjVCpVPj666/tXZVqyczMxCOPPIJGjRpBpVJh5syZVy2bn5+PESNGICwsDCqVCmPHjq3Rup5++mk0b97cappKpcJrr71W43rTZZbP3MaNG+1dFbur6jPm6g4dOoQ+ffrAz88PKpUK33//vb2rVKU777wTd955p72rcU01+b50BK+99hpUKpW9q3FdtvxddbNBfW44CxcuxNChQ6HT6XDkyBE0adLEav6dd96J7Oxs7Nmzx041dBzjxo3D2rVrMXXqVISFhaFbt25XLfv2229j4cKFePXVVxEVFYV27do1YE2JqDYSExORnp6Ot956C/7+/tfcx+vbvn37sGzZMocNrzX5vqSa+/LLL5GVlVXj/3DXhFOGIovi4mLMmDEDH330kb2r4rDWr1+PBx98EP/+97+rVbZXr16YOnVqA9SMiOqqsLAQKSkpePnllzFq1Ch7Vwf79u3DtGnTcOedd1YKRT///LN9KlUDNfm+pJr78ssvsWfPnnoNRU59+qxr16749NNPcebMGXtXpcEVFBTYZDlZWVnw9/e3eVm6PrPZjKKiIntXg5zYuXPnAMAh9lutVgutVmvvalwTvwMdn1OHopdeegkmkwkzZsy4Zrljx45BpVJh4cKFleZd2afFco7177//xpNPPgk/Pz8EBwfj1VdfhYjg5MmTePDBB6HX6xEWFob33nuvynWaTCa89NJLCAsLg7e3Nx544AGcPHmyUrktW7bgvvvug5+fH7y8vNC7d2/88ccfVmUsddq3bx+eeOIJBAQE4LbbbrvmNh89ehSPPvooAgMD4eXlhV69emHVqlXK/IULF0KlUkFEMHv2bKhUqqueW7acz01PT8eqVauUsseOHVOWc+zYsSpfU5N+Lhs2bIBKpcJ3331Xad6XX34JlUqFlJSUay4jNzcX48aNQ/PmzaHT6RAREYEhQ4YgOztbKZOVlYXhw4cjNDQUHh4e6NKlCz7//HNlfmlpKQIDAzF06NBKyzcajfDw8LD6n2JxcTGmTp2KVq1aQafTITIyEhMnTkRxcbHVa1UqFUaNGoXFixejQ4cO0Ol0WLNmDQDg9OnTGDZsGEJDQ6HT6dChQwf83//9X6X1nzp1Cv3794e3tzdCQkIwbty4Suu5luutp7CwENHR0YiOjkZhYaEyPScnB40bN8Ytt9wCk8kEANi1axeefvpptGzZEh4eHggLC8OwYcNw/vx5q3XWdZ+yfJaWLl1arX3qSmazGTNnzkSHDh3g4eGB0NBQPPfcc7hw4YJVue3btyM+Ph5BQUHw9PREixYtMGzYsOsu/4cffkBCQgLCw8Oh0+kQFRWFN954Q2kni0OHDmHAgAEICwuDh4cHIiIi8Pjjj8NgMFxz+b/99hseffRRNG3aVPl8jRs3zur9qcprr72GZs2aAQAmTJgAlUqlHJ252umrqvqYWD6333//PTp27Kh8biyf3YpOnz6N4cOHK23RokULPP/88ygpKcHChQvx6KOPAgDuuusu5XvE8h1RVZ+i6+2rwOXv9//+97+YP38+oqKioNPp0L17d2zbtu2abWRhy+/LK+s0e/ZstGzZEl5eXujTpw9OnjwJEcEbb7yBiIgIeHp64sEHH0ROTo7VMqrzudq/fz88PT0xZMgQq9f+/vvv0Gg0mDRpUrW2/0pffPEFYmJi4OnpicDAQDz++OOV9rU777wTHTt2xL59+3DXXXfBy8sLTZo0wTvvvFNpecePH8cDDzxg9b21du3aSu//qlWrcPz4caV9r/yMms1mvPXWW4iIiICHhwfuueceHD58uGYbJ07os88+EwCybds2GTZsmHh4eMjp06eV+b1795YOHTooz9PT0wWAfPbZZ5WWBUCmTp2qPJ86daoAkK5du8qgQYNkzpw5kpCQIADk/fffl7Zt28rzzz8vc+bMkVtvvVUAyK+//qq8fsOGDQJAOnXqJJ07d5b3339fXnzxRfHw8JA2bdrIxYsXlbLJycmi1WolNjZW3nvvPfnggw+kc+fOotVqZcuWLZXq1L59e3nwwQdlzpw5Mnv27Ku2T0ZGhoSGhoqvr6+8/PLL8v7770uXLl1ErVbLt99+KyIiR44ckUWLFgkAuffee2XRokWyaNGiqy5v0aJFEhQUJF27dlXK5ufnK+9Fenq61Wss7bBhwwZlWmJiojRr1uyq7W82myUyMlIGDBhQqQ79+vWTqKioq26ziEheXp507NhRNBqNPPPMMzJ37lx54403pHv37rJz504REbl48aK0a9dO3N3dZdy4cfLhhx/K7bffLgBk5syZyrKGDRsm/v7+UlxcbLWOzz//XPnsiYiYTCbp06ePeHl5ydixY+WTTz6RUaNGiZubmzz44IOVtrVdu3YSHBws06ZNk9mzZ8vOnTslIyNDIiIiJDIyUl5//XWZO3euPPDAAwJAPvjgA+X1Fy9elDZt2oiHh4dMnDhRZs6cKTExMdK5c+dKbV2V6q5n8+bNotFoZNy4ccq0xx9/XDw9PeXgwYPKtP/+979y++23y+uvvy7z58+XMWPGiKenp/To0UPMZrNSriH3qao+YyNGjBA3Nzd55plnZN68eTJp0iTx9vaW7t27S0lJiYiIZGZmSkBAgLRp00beffdd+fTTT+Xll1+Wdu3aXbNNRUT69+8vjz32mLz77rsyd+5cefTRRwWA/Pvf/1bKFBcXS4sWLSQ8PFzefPNNWbBggUybNk26d+8ux44du+byX3jhBenXr5+8/fbb8sknn8jw4cNFo9HII488cs3X/fXXX/LBBx8IABk0aJAsWrRIvvvuu6u2k8jl96oiANKlSxdp3LixvPHGGzJz5kxp2bKleHl5SXZ2tlLu9OnTEh4eruwL8+bNk1dffVXatWsnFy5ckCNHjsjo0aMFgLz00kvK90hGRoaIlH9v9+7dW1ledfdVy/f7TTfdJK1atZL//Oc/8s4770hQUJBEREQo7/HV2Pr7smKdunbtKu3bt5f3339fXnnlFdFqtdKrVy956aWX5JZbbpEPP/xQRo8eLSqVSoYOHWq1jOp8rkRE3n33XQEgP/zwg4iI5OfnS1RUlLRv316Kioquue1Vvd9vvvmmqFQqGThwoMyZM0emTZsmQUFB0rx5c7lw4YJSrnfv3hIeHi6RkZEyZswYmTNnjtx9990CQFavXq2Uy8/Pl5YtW4qnp6e8+OKLMnPmTOnRo4d06dLF6nvr559/lq5du0pQUJDSvpbPq+U74KabbpKYmBj54IMP5LXXXhMvLy/p0aPHNbfxSk4fio4cOSJubm4yevRoZb4tQtGzzz6rTCsrK5OIiAhRqVQyY8YMZfqFCxfE09NTEhMTlWmWN69JkyZiNBqV6cuWLRMAMmvWLBEpDwCtW7eW+Ph4qx+QixcvSosWLeTee++tVKdBgwZVq33Gjh0rAOS3335TpuXl5UmLFi2kefPmYjKZrLY/KSmpWstt1qyZJCQkWE2zZSgSEZk8ebLodDrJzc1VpmVlZYmbm5tVuapMmTJFAChfZBVZ2njmzJkCQL744gtlXklJicTGxoqPj4/ynq1du1YAyIoVK6yW069fP2nZsqXyfNGiRaJWq63aWkRk3rx5AkD++OMPq21Vq9Wyd+9eq7LDhw+Xxo0bW/3AiJQHET8/P+VH31L3ZcuWKWUKCgqkVatW1QpF1V2PSPn7oFarZdOmTbJ8+fJKP0QiYlXe4quvvhIAsmnTJmVaQ+1TIpU/Y7/99psAkMWLF1vVc82aNVbTv/vuO6uwWxNVtcNzzz0nXl5eyo/Szp07BYAsX77cJsufPn26qFQqOX78+DVfa/nue/fdd62m1zQUabVaOXz4sDLtr7/+EgDy0UcfKdOGDBkiarW6yja07H+Wz1JVn9UrQ1F191XLNjZq1EhycnKUsj/88EOV+/CV6uP70lKn4OBgq++yyZMnKyGztLRUmT5o0CDRarVWIaY6nyuR8v+Y3XbbbRIaGirZ2dmSlJQkbm5u1fosX/l+Hzt2TDQajbz11ltW5Xbv3i1ubm5W03v37i0A5H//+58yrbi4WMLCwqz+Y/vee+8JAPn++++VaYWFhRIdHV3ps5CQkFDl59LyHdCuXTur/6jOmjVLAMju3buvu60WTn36DABatmyJp556CvPnz8fZs2dtttwRI0Yof2s0GnTr1g0iguHDhyvT/f390bZtWxw9erTS64cMGQJfX1/l+SOPPILGjRtj9erVAIC0tDQcOnQITzzxBM6fP4/s7GxkZ2ejoKAA99xzDzZt2gSz2Wy1zJEjR1ar7qtXr0aPHj2sTrH5+Pjg2WefxbFjx7Bv377qNYIdDBkyBMXFxVZDL5cuXYqysjI8+eST13ztN998gy5duuChhx6qNM9yqHv16tUICwvDoEGDlHnu7u4YPXo08vPz8euvvwIA7r77bgQFBWHp0qVKuQsXLmDdunUYOHCgMm358uVo164doqOjlfcwOzsbd999N4DyU4IV9e7dG+3bt1eeiwi++eYb3H///RARq2XEx8fDYDBgx44dSt0bN26MRx55RHm9l5cXnn322Wu2S03XA5SfRunQoQMSExPxz3/+E71798bo0aOtlunp6an8XVRUhOzsbPTq1QsArJZlUd/7VFWWL18OPz8/3HvvvVbbHBMTAx8fH+X9sfQTWblyJUpLS6+6vKpUbIe8vDxkZ2fj9ttvx8WLF3HgwAEAgJ+fHwBg7dq1uHjxYq2XX1BQgOzsbNxyyy0QEezcubNGy6qtuLg4REVFKc87d+4MvV6vvE9msxnff/897r///ipHZNVm2Hd191WLgQMHIiAgQHl+++23A0CVn6Ur11Nf35ePPvqo8t4DQM+ePQEATz75JNzc3Kyml5SU4PTp08q06nyuAECtVmPhwoXIz89H3759MWfOHEyePLlWI+O+/fZbmM1mPPbYY1b7S1hYGFq3bl3p+8zHx8fqe1mr1aJHjx5Wbb5mzRo0adIEDzzwgDLNw8MDzzzzTI3rN3ToUKt+Z9V9jyty+lAEAK+88grKysqu27eoJpo2bWr13M/PDx4eHggKCqo0/cq+CQDQunVrq+cqlQqtWrVS+t4cOnQIQPlw2eDgYKvHggULUFxcXKmvQYsWLapV9+PHj6Nt27aVpluG0B8/frxay7GH6OhodO/eHYsXL1amLV68GL169UKrVq2u+dojR46gY8eO1yxz/PhxtG7dGmq19a5xZdu4ublhwIAB+OGHH5Q+O99++y1KS0utQtGhQ4ewd+/eSu9hmzZtAJT3iajoyvfw3LlzyM3Nxfz58ystw9KnybKM48ePo1WrVpV+YKp6r69Uk/UA5V9u//d//4f09HTk5eXhs88+q7TenJwcjBkzBqGhofD09ERwcLCyfVX1k6nvfaoqhw4dgsFgQEhISKXtzs/PV7a5d+/eGDBgAKZNm4agoCA8+OCD+Oyzz6rVX2vv3r146KGH4OfnB71ej+DgYOWHwtIOLVq0wPjx47FgwQIEBQUhPj4es2fPvm5/IgA4ceIEnn76aQQGBsLHxwfBwcHo3bu31fLr25XvHQAEBAQo79O5c+dgNBqvu//VRHX31avV0RKQqvosXbme+vq+rOozDwCRkZFVTq9Y1+p8riyioqLw2muvYdu2bejQoQNeffXVWtX30KFDEBG0bt260v6yf//+St9nERERlb4XKn4ugPL2i4qKqlTuet/nVante1yRUw/Jt2jZsiWefPJJzJ8/Hy+++GKl+Vf7X8qVHSEr0mg01ZoGlP8vvKYsR4HeffdddO3atcoyPj4+Vs8r/s/hRlGbtr2eIUOGYMyYMTh16hSKi4uxefNmfPzxx7VeXm09/vjj+OSTT/DTTz+hf//+WLZsGaKjo9GlSxeljNlsRqdOnfD+++9XuYwrv/yufA8tn4Mnn3wSiYmJVS6jc+fOddmMWq9n7dq1AMqPAh06dKhSoHvsscfw559/YsKECejatSt8fHxgNptx3333VTrKCdT/PlUVs9mMkJAQq5BdUXBwMAAoF4bbvHkzVqxYgbVr12LYsGF47733sHnz5kr7okVubi569+4NvV6P119/HVFRUfDw8MCOHTswadIkq3Z477338PTTT+OHH37Azz//jNGjR2P69OnYvHkzIiIiqly+yWTCvffei5ycHEyaNAnR0dHw9vbG6dOn8fTTT1fZztVR0/22vt8nW7gR63i1Ol2vrjX5XFlYLmlw5swZnD9/HmFhYTWur9lshkqlwk8//VRlHa/cDxq6zW2xPpcIRUD50aIvvvgC//nPfyrNs6TJ3Nxcq+n1ecTEciTIQkRw+PBh5YfHcihar9cjLi7Oputu1qwZDh48WGm65ZCrZUSKLdRH2z7++OMYP348vvrqKxQWFsLd3d3q6MzVREVFXfeCnc2aNcOuXbtgNput/gdaVdvccccdaNy4MZYuXYrbbrsN69evx8svv1xpnX/99RfuueeeWp0iCA4Ohq+vL0wm03U/B82aNcOePXsgIlbrquq9rst6gPKRZa+//jqGDh2KtLQ0jBgxArt377b6H21ycjKmTZuGKVOmKK+78nNvS9fbp6oSFRWFX375Bbfeemu1/lPRq1cv9OrVC2+99Ra+/PJLDB48GEuWLLE69VfRxo0bcf78eXz77be44447lOnp6elVlu/UqRM6deqEV155BX/++SduvfVWzJs3D2+++WaV5Xfv3o2///4bn3/+udUIo3Xr1l13W64lICCg0j4L1H6/DQ4Ohl6vv+7+V5N9pCb7al005PdlddX0czVv3jysW7cOb731FqZPn47nnnsOP/zwQ43XGxUVBRFBixYtlKPdddWsWTPs27ev0vdWVaPGGuLq2i5x+gwofzOffPJJfPLJJ8jIyLCap9frERQUhE2bNllNnzNnTr3V53//+x/y8vKU519//TXOnj2Lvn37AgBiYmIQFRWF//73v8jPz6/0esv1RWqjX79+2Lp1q9Xw9YKCAsyfPx/Nmze36tNSV5ZwV7FtTSYT5s+fX+tlBgUFoW/fvvjiiy+wePFi3HfffZVOsVRlwIAB+Ouvv6oc0m/5n0S/fv2QkZFh1VeorKwMH330EXx8fJTTEkD5ufpHHnkEK1aswKJFi1BWVlYpnD322GM4ffo0Pv3000rrLCwsvO71pDQaDQYMGIBvvvmmyh+Uip+Dfv364cyZM1b9rS5evFittq7JekpLS/H0008jPDwcs2bNwsKFC5GZmYlx48ZZLQ+o/D+0+rztwfX2qao89thjMJlMeOONNyrNKysrU4LBhQsXKm2L5QjutU6hVdUOJSUllb5bjEYjysrKrKZ16tQJarW6xssXEcyaNeuqr6mOqKgoGAwG7Nq1S5l29uzZKved6lCr1ejfvz9WrFiB7du3V5pvqb+3tzeAyv+JqkpN9tW6aMjvy+qq7ucKKA9KEyZMwIABA/DSSy/hv//9L3788Uf873//q/F6H374YWg0GkybNq3S/iAilS63UR3x8fE4ffo0fvzxR2VaUVFRld+Z3t7e9X5K2GWOFAHAyy+/jEWLFuHgwYPo0KGD1bwRI0ZgxowZGDFiBLp164ZNmzbh77//rre6BAYG4rbbbsPQoUORmZmJmTNnolWrVkrnMrVajQULFqBv377o0KEDhg4diiZNmuD06dPYsGED9Ho9VqxYUat1v/jii/jqq6/Qt29fjB49GoGBgfj888+Rnp6Ob775ptI5+rro0KEDevXqhcmTJyMnJweBgYFYsmRJpR+AmhoyZIjSobiqH7SqTJgwAV9//TUeffRRDBs2DDExMcjJycGPP/6IefPmoUuXLnj22WfxySef4Omnn0ZqaiqaN2+Or7/+Gn/88Qdmzpxp1ZEXKO+8+dFHH2Hq1Kno1KlTpVubPPXUU1i2bBlGjhyJDRs24NZbb4XJZMKBAwewbNkyrF279rodHmfMmIENGzagZ8+eeOaZZ9C+fXvk5ORgx44d+OWXX5TrlzzzzDP4+OOPMWTIEKSmpqJx48ZYtGgRvLy8qtU+1V3Pm2++ibS0NCQnJ8PX1xedO3fGlClT8Morr+CRRx5Bv379oNfrcccdd+Cdd95BaWkpmjRpgp9//vmq/5O1hevtU1Xp3bs3nnvuOUyfPh1paWno06cP3N3dcejQISxfvhyzZs3CI488gs8//xxz5szBQw89hKioKOTl5eHTTz+FXq9Hv379rrr8W265BQEBAUhMTMTo0aOhUqmwaNGiSj8o69evx6hRo/Doo4+iTZs2KCsrw6JFi5SwejXR0dGIiorCv//9b5w+fRp6vR7ffPNNjfpQVOXxxx/HpEmT8NBDD2H06NG4ePEi5s6dizZt2lTZSb463n77bfz888/o3bs3nn32WbRr1w5nz57F8uXL8fvvv8Pf3x9du3aFRqPBf/7zHxgMBuh0Otx9990ICQmptLya7qu11ZDfl9VV3c+ViGDYsGHw9PTE3LlzAQDPPfccvvnmG4wZMwZxcXEIDw+v9nqjoqLw5ptvYvLkyTh27Bj69+8PX19fpKen47vvvsOzzz5b46t5P/fcc/j4448xaNAgjBkzBo0bN8bixYvh4eEBwProUExMDJYuXYrx48eje/fu8PHxwf3331+j9V1XtcepOZCKQ/KvlJiYKACshuSLlA9vHD58uPj5+Ymvr6889thjkpWVddUh+efOnau0XG9v70rru3L4v2Xo4FdffSWTJ0+WkJAQ8fT0lISEhCqHz+7cuVMefvhhadSokeh0OmnWrJk89thjkpycfN06XcuRI0fkkUceEX9/f/Hw8JAePXrIypUrK5VDHYfkW9YVFxcnOp1OQkND5aWXXpJ169bVaki+RXFxsQQEBIifn58UFhZWq34iIufPn5dRo0ZJkyZNRKvVSkREhCQmJloNQ8/MzJShQ4dKUFCQaLVa6dSpU5WXaxC5fO0kAPLmm29WWaakpET+85//SIcOHUSn00lAQIDExMTItGnTxGAwWG3r1do6MzNTkpKSJDIyUtzd3SUsLEzuuecemT9/vlW548ePywMPPCBeXl4SFBQkY8aMUYaXX29IfnXWk5qaKm5ubvLCCy9Yva6srEy6d+8u4eHhyrVKTp06JQ899JD4+/uLn5+fPProo3LmzBm77lNXG2o+f/58iYmJEU9PT/H19ZVOnTrJxIkT5cyZMyIismPHDhk0aJA0bdpUdDqdhISEyD/+8Q/Zvn37ddv0jz/+kF69eomnp6eEh4fLxIkTlUs6WN6To0ePyrBhwyQqKko8PDwkMDBQ7rrrLvnll1+uu/x9+/ZJXFyc+Pj4SFBQkDzzzDPKkPirfW4trjYkX6T8ujAdO3YUrVYrbdu2lS+++OKqQ/Kr+tw2a9bM6tIJIuWfzyFDhkhwcLDodDpp2bKlJCUlWQ2j/vTTT6Vly5ai0Wis2ujKIfki1dtXr7WNV/t+uZKtvy+vVifLZ/nKSzNU9ZtWnc+VZUj6N998Y7W8EydOiF6vl379+l2znlW93yIi33zzjdx2223i7e0t3t7eEh0dLUlJSVbXKbtyP7Woah88evSoJCQkiKenpwQHB8u//vUv+eabbwSAbN68WSmXn58vTzzxhPj7+wsAZTlXa7drXW7nalQiN1BPOKJqKisrQ3h4OO6//378v//3/+xdHbKjjRs34q677sLy5cutLkdARI5r5syZGDduHE6dOlXppu71yWX6FJFz+f7773Hu3LlKl68nIiLHcuUtaYqKivDJJ5+gdevWDRqIABfrU0SOb8uWLdi1axfeeOMN3HTTTTbrTElERPbx8MMPo2nTpujatSsMBgO++OILHDhw4KqXyqhPDEXkUObOnYsvvvgCXbt2rfIGvkRE5Fji4+OxYMECLF68GCaTCe3bt8eSJUuqdakVW2OfIiIiIiKwTxERERERAIYiIiIiIgAu3qfIbDbjzJkz8PX1bZDLhxMREVHdiQjy8vIQHh5u0wtounQoOnPmTKUbchIREZFjOHny5FVvmFwbLh2KLJeBP3nyJPR6vZ1rQ9VWUABYLk1/5gxw6X5JRETkGoxGIyIjI212OxcLlw5FllNmer2eociRXLoZIgBAr2coIiJyUbbu+sKO1kRERERgKCIiIiICwFBEREREBIChiIiIiAgAQxERERERAIYiIiIiIgAMRUREREQAGIqIiIiIADAUEREREQFgKCIiIiICwFBEREREBIChiIiIiAiAi98QloiqR0RQZhaUmQQlJjNKLz3MApjNArNI+d8ikEt/my5Nl0vTTeYKDxGYzYCgvKyIQABALk0zA4LL0+XScsr/tn4dKk674nWosMxLRZVllP9tPd2yMLn85+V1WL3u8nRULIuK9al6esU2vV5ZqzVUqMeVy6ouQc1fVLv11OI1tXkRardNDfSSSp+R6r2mFuup+Usa7PMDAOPvbQNfD/davbahMRQR3eAsgaSo1ITiMjOKSk0oKjWjuOzyv8WlZqv5VytXVGpG0aXy5c8rv6akzIyyS+GlzCQoM5eHHyKi2nj+ziiGIiK6TERQUGLChYIS5BSUIOdiifL3hYslyCkohaGwBIbCUhgLy5BfXP4oKC5DUanphgwl7hoV1CrLA1CrK/ytUkGlUkGjhlJGpQI0alX5o8I0lUoFFXDp70uvRfkEFQD1lWWgUspW/FutUqH8ZRXLotK0S1OUvytOv7RmZZnKNKuyqipeZ70ey0Sr5SllrKdf/reaZSvUoTZq89IKW1XP66mdWq2rFi+qVf2crL1rsx4vreNEDcepKdENqqTMjHP5xcgwFCHTWHT53wp/ZxqLUVhqssn6dG5q6NzU8HDXQOeuhoebpvzvS9M83NXQuZXP07mVP7ea76aGrkI5j0vLsJTXuanhplHD7VKAufyvGhqNClqNGu4aVZ1+mImIbkQMRUTVVGYy42h2AfadMWLvGQP2nTXicFY+Mo3F1V6Gh7sagV5aBHhrEeitRYDX5X/9vdzh5+kOXw83+Hq4w0fnBm+d5lKQKQ8tWo0aajXDCBFRfWAoIqpCQXEZDmQYse+MEfvOGrH3jBEHM/JQXGausry7RoUQXw+E6nUI8/NAqN4DYXoP5e9QvQdCfHXw1nGXIyK6UfEbmlyeiOBodgE2Hz2PXScN2HY8B+nZBVWOzvDWahDdWI8O4Xq0b6xHmzBfNAv0QoCXlkdwiIgcHEMRuaRMYxH+OJyNPw6fx59HsnHWUFSpTIivDh3C9egQ7of2l0JQ00Avhh8iIifFUEQuwWwWpJ3Kxbp9mUjen4m/M/Ot5ms1asQ0C8BNTf1xc9MAdG3qjyAfnZ1qS0RE9sBQRE7rYkkZ/jx8HskHMvHL/iycy7vcIVqlAjo18cMtUUG4tVUjdGsWCE+txo61JSIie2MoIqciIkg5ch4L/zyGDQezUGq63DHIR+eGO9sG4972oejdJhj+Xlo71pSIiG40Nbr32WuvvVZ+cbIKj+joaGV+UVERkpKS0KhRI/j4+GDAgAHIzMy0WsaJEyeQkJAALy8vhISEYMKECSgrK7Mqs3HjRtx8883Q6XRo1aoVFi5cWKkus2fPRvPmzeHh4YGePXti69atNdkUcjKFJSYs3nIc8TM34YkFW/DzvkyUmgQRAZ4YEtsM/xvWAztevRcfP3EzHuzahIGIiIgqqfGRog4dOuCXX365vAC3y4sYN24cVq1aheXLl8PPzw+jRo3Cww8/jD/++AMAYDKZkJCQgLCwMPz55584e/YshgwZAnd3d7z99tsAgPT0dCQkJGDkyJFYvHgxkpOTMWLECDRu3Bjx8fEAgKVLl2L8+PGYN28eevbsiZkzZyI+Ph4HDx5ESEhInRqEHEumsQj/t+kElmw9CUNhKQDAS6vBgJsj8GSvZmgT6sOLDBIRUbWopAZ3rHvttdfw/fffIy0trdI8g8GA4OBgfPnll3jkkUcAAAcOHEC7du2QkpKCXr164aeffsI//vEPnDlzBqGhoQCAefPmYdKkSTh37hy0Wi0mTZqEVatWYc+ePcqyH3/8ceTm5mLNmjUAgJ49e6J79+74+OOPAQBmsxmRkZF44YUX8OKLL161/sXFxSguvtyvxGg0IjIyEgaDAXq9vrrNQPZWUAD4+AAAuk74Frnq8qM+TQO9kHhLczzaLQJ6B7nPDhER1ZzRaISfn5/Nf79rdPoMAA4dOoTw8HC0bNkSgwcPxokTJwAAqampKC0tRVxcnFI2OjoaTZs2RUpKCgAgJSUFnTp1UgIRAMTHx8NoNGLv3r1KmYrLsJSxLKOkpASpqalWZdRqNeLi4pQyVzN9+nT4+fkpj8jIyJpuPtlZSZkZ8zcdUZ4Xl5kR0ywAnw7phg3/vhPDb2vBQERERLVSo1DUs2dPLFy4EGvWrMHcuXORnp6O22+/HXl5ecjIyIBWq4W/v7/Va0JDQ5GRkQEAyMjIsApElvmWedcqYzQaUVhYiOzsbJhMpirLWJZxNZMnT4bBYFAeJ0+erMnmk51tO5aDf3z0Gz5Yd0iZNu+pm/H1yFjc2z4UGl4/iIiI6qBGfYr69u2r/N25c2f07NkTzZo1w7Jly+Dp6WnzytmaTqeDTsdrzzgaw8VSvL16P5ZuLw+xTbwvHwnq3SakdrdtJiIiukKNT59V5O/vjzZt2uDw4cMICwtDSUkJcnNzrcpkZmYiLCwMABAWFlZpNJrl+fXK6PV6eHp6IigoCBqNpsoylmWQ89hz2oD7P/5dCUSDekRi1ejb7VwrIiJyRnUKRfn5+Thy5AgaN26MmJgYuLu7Izk5WZl/8OBBnDhxArGxsQCA2NhY7N69G1lZWUqZdevWQa/Xo3379kqZisuwlLEsQ6vVIiYmxqqM2WxGcnKyUoacw8I/0vHQnD9wIuciIgM98fXIWEx/uDOH0xMRUf2QGvjXv/4lGzdulPT0dPnjjz8kLi5OgoKCJCsrS0RERo4cKU2bNpX169fL9u3bJTY2VmJjY5XXl5WVSceOHaVPnz6SlpYma9askeDgYJk8ebJS5ujRo+Ll5SUTJkyQ/fv3y+zZs0Wj0ciaNWuUMkuWLBGdTicLFy6Uffv2ybPPPiv+/v6SkZFRk80Rg8EgAMRgMNTodVT/Pkr+W5pNWinNJq2UZz7fJhcKii/PzM8XAcof+fn2qyQREdlFff1+16hP0alTpzBo0CCcP38ewcHBuO2227B582YEBwcDAD744AOo1WoMGDAAxcXFiI+Px5w5c5TXazQarFy5Es8//zxiY2Ph7e2NxMREvP7660qZFi1aYNWqVRg3bhxmzZqFiIgILFiwQLlGEQAMHDgQ586dw5QpU5CRkYGuXbtizZo1lTpfk2OaveEw/vvz3wCACfFt8c87o3itISIiqnc1uk6Rs6mv6xxQ7X266SjeWr0fQHkgSrqrVeVCFa5ThPx8wNu7AWtIRET2dsNcp4iovqzefVYJRP+6t03VgYiIiKieMBTRDWHz0fMYuzQNADD01uZ44Z7W9q0QERG5HIYisrv07AI887/tKCkzo0/7ULyS0N7eVSIiIhfEUER2VVRqQtLiHcgrKkNMswB8OOgmXpmaiIjsgqGI7Gr66v3Yd9aIAC93zH7iZni4a+xdJSIiclEMRWQ3P+0+i89TjgMA3h/YFWF+HnauERERuTKGIrKL07mFmPjNLgDAc71b4q62IXauERERuTqGIrKL137ci7yiMtzU1B//7tPW3tUhIiJiKKKG98u+TKzblwk3tQr/GdAZ7hp+DImIyP74a0QN6mJJGab+uBcAMOL2lmgT6mvnGhEREZVjKKIGNXvDYZzOLUQTf0+MvodXrCYiohsHQxE1mDO5hVjwWzoA4NV/tIeXtkb3IyYiIqpXDEXUYGb+8jeKy8zo0TwQ8R1C7V0dIiIiKwxF1CD+zszD16mnAAAv9ouGSsWrVhMR0Y2FoYgaxH9+OgCzAH07huHmpgH2rg4REVElDEVU77am5yD5QBY0ahX+Hc9rEhER0Y2JoYjq3YfJhwAAA7tHIirYx861ISIiqhpDEdWrv07m4vfD2XBTq/DPO6PsXR0iIqKrYiiiejVn42EAwINdmyAiwMvOtSEiIro6hiKqNydzLuLnfZkAgJG9W9q5NkRERNfGUET15n8pxyAC3NEmGK15Ow8iIrrBMRRRvSgqNWHZ9vLrEj19SzM714aIiOj6GIqoXqzdmwFDYSma+HvizjYh9q4OERHRdTEUUb1Ytv0kAOCRmAio1bx6NRER3fgYisjmTuZcxB+Hz0OlKg9FREREjoChiGxu+aV7nN0aFYTIQA7DJyIix8BQRDZlNgu+uRSKHuseaefaEBERVR9DEdlU6okLOJ1bCF+dG/q0D7V3dYiIiKqNoYhsasVfZwAAfTqEwcNdY+faEBERVR9DEdlMmcmM1bvPAgD+0aWxnWtDRERUMwxFZDNb0nOQnV8Cfy933NYqyN7VISIiqhGGIrIZy6mzvh0bw13DjxYRETkW/nKRTZSUmbFmbwYA4P7OPHVGRESOh6GIbOKPw9nIvViKIB8derZsZO/qEBER1RhDEdnEil3lp87+0bkxNLytBxEROSCGIqozk1mw4UAWAOC+jmF2rg0REVHtMBRRne06lYsLF0vhq3NDTLMAe1eHiIioVhiKqM42HjwHALitdRBHnRERkcPiLxjV2ca/y0PRnW2D7VwTIiKi2mMoojrJKSjBrlO5AIDebULsWxkiIqI6YCiiOtn09zmIANFhvgjz87B3dYiIiGqNoYjqZOPB8lFnd0XzKBERETk2hiKqNbNZsOlQNgDgzjbsT0RERI6NoYhqbc8ZA3IKSuCrc8PNHIpPREQOjqGIam3z0fMAgJ4tAzkUn4iIHB5/yajWNh/NAQD04r3OiIjICTAUUa2UmczYms5QREREzoOhiGpl7xkj8ovLoPdwQ7vGentXh4iIqM4YiqhWUi71J+rRohE0apWda0NERFR3DEVUK5ZO1rFRPHVGRETOgaGIaqzMZMY2pT9RoJ1rQ0REZBsMRVRju08bUFBigp+nO9qFsT8RERE5B4YiqjHLUPyeLQKhZn8iIiJyEnUKRTNmzIBKpcLYsWOVaUVFRUhKSkKjRo3g4+ODAQMGIDMz0+p1J06cQEJCAry8vBASEoIJEyagrKzMqszGjRtx8803Q6fToVWrVli4cGGl9c+ePRvNmzeHh4cHevbsia1bt9Zlc6iaLP2JOBSfiIicSa1D0bZt2/DJJ5+gc+fOVtPHjRuHFStWYPny5fj1119x5swZPPzww8p8k8mEhIQElJSU4M8//8Tnn3+OhQsXYsqUKUqZ9PR0JCQk4K677kJaWhrGjh2LESNGYO3atUqZpUuXYvz48Zg6dSp27NiBLl26ID4+HllZWbXdJKqGUpMZ246VHyliJ2siInIqUgt5eXnSunVrWbdunfTu3VvGjBkjIiK5ubni7u4uy5cvV8ru379fAEhKSoqIiKxevVrUarVkZGQoZebOnSt6vV6Ki4tFRGTixInSoUMHq3UOHDhQ4uPjlec9evSQpKQk5bnJZJLw8HCZPn16tbfDYDAIADEYDNXfeBeXejxHmk1aKV2mrRWTyWyfSuTniwDlj/x8+9SBiIjspr5+v2t1pCgpKQkJCQmIi4uzmp6amorS0lKr6dHR0WjatClSUlIAACkpKejUqRNCQ0OVMvHx8TAajdi7d69S5splx8fHK8soKSlBamqqVRm1Wo24uDilTFWKi4thNBqtHlQzKUcu3e+M/YmIiMjJuNX0BUuWLMGOHTuwbdu2SvMyMjKg1Wrh7+9vNT00NBQZGRlKmYqByDLfMu9aZYxGIwoLC3HhwgWYTKYqyxw4cOCqdZ8+fTqmTZtWvQ2lKinXJ2J/IiIicjI1OlJ08uRJjBkzBosXL4aHh0d91aneTJ48GQaDQXmcPHnS3lVyKKUmM7YfuwAA6MX+RERE5GRqFIpSU1ORlZWFm2++GW5ubnBzc8Ovv/6KDz/8EG5ubggNDUVJSQlyc3OtXpeZmYmwsDAAQFhYWKXRaJbn1yuj1+vh6emJoKAgaDSaKstYllEVnU4HvV5v9aDq23UqF4WlJgR6a9EmxNfe1SEiIrKpGoWie+65B7t370ZaWpry6NatGwYPHqz87e7ujuTkZOU1Bw8exIkTJxAbGwsAiI2Nxe7du61Gia1btw56vR7t27dXylRchqWMZRlarRYxMTFWZcxmM5KTk5UyZHu8PhERETmzGvUp8vX1RceOHa2meXt7o1GjRsr04cOHY/z48QgMDIRer8cLL7yA2NhY9OrVCwDQp08ftG/fHk899RTeeecdZGRk4JVXXkFSUhJ0Oh0AYOTIkfj4448xceJEDBs2DOvXr8eyZcuwatUqZb3jx49HYmIiunXrhh49emDmzJkoKCjA0KFD69QgdHWWofg9WvDWHkRE5Hxq3NH6ej744AOo1WoMGDAAxcXFiI+Px5w5c5T5Go0GK1euxPPPP4/Y2Fh4e3sjMTERr7/+ulKmRYsWWLVqFcaNG4dZs2YhIiICCxYsQHx8vFJm4MCBOHfuHKZMmYKMjAx07doVa9asqdT5mmxDRLD7lAEA0DXS376VISIiqgcqERF7V8JejEYj/Pz8YDAY2L/oOk7nFuLWGevhplZhz7R4eLhr7FeZggLAx6f87/x8wNvbfnUhIqIGV1+/37z3GVXL7lO5AIC2Yb72DURERET1hKGIquWvS6fOOkf42bkmRERE9YOhiKrF0p+oUxN/+1aEiIionjAU0XWJCHZdOn3GI0VEROSsGIrouk7kXISxqAxaNzXahPKijURE5JwYiui6LP2J2jXWQ+vGjwwRETkn/sLRdVlGnnVuwlNnRETkvBiK6Lp2ceQZERG5AIYiuiazWbDntCUU+du3MkRERPWIoYiu6Wh2PgpKTPB01yAqmFeOJiIi58VQRNdkOXXWIVwPNw0/LkRE5Lz4K0fXdLk/kb99K0JERFTPGIromnafZidrIiJyDQxFdFVlJjP2nrl0ew+GIiIicnIMRXRVh7LyUVRqhq/ODS0asZM1ERE5N4YiuirLTWA7NvGDWq2yc22IiIjqF0MRXdWu07kA2J+IiIhcA0MRXZXlSBH7ExERkStgKKIqlZSZsf9sHgCgC4fjExGRC2AooiodzMhDickMfy93RAR42rs6RERE9Y6hiKpk6U/UqYkfVCp2siYiIufHUERV2n2KF20kIiLXwlBEVfqLt/cgIiIXw1BElRSVmvB3Znknax4pIiIiV8FQRJXsP2uEySwI8tEiTO9h7+oQERE1CIYiqsQyFL9dYz07WRMRkctgKKJK9p81AgDaN9bbuSZEREQNh6GIKrGEonYMRURE5EIYisiK2Sw4kHH59BkREZGrYCgiKycvXER+cRm0GjVaBnvbuzpEREQNhqGIrFhOnbUO9YG7hh8PIiJyHfzVIyv7zvLUGRERuSaGIrJygJ2siYjIRTEUkZWDl65k3S7M1841ISIialgMRaQoKC7D8fMXAQBtGYqIiMjFMBSRwnKUKNhXh0Y+OjvXhoiIqGExFJHi4KXrE0XzKBEREbkghiJSWDpZMxQREZErYigixQHlSBFHnhERkethKCIAgMjl23tEN+aRIiIicj0MRQQAyDAWwVBYCo1ahVYhPvauDhERUYNjKCIAl0+dtQzyhs5NY+faEBERNTyGIgIAHDhrOXXG/kREROSaGIoIAHAggyPPiIjItTEUEQBeo4iIiIihiFBSZsbhrHwAvL0HERG5LoYiwpFz+SgzC3x1bmji72nv6hAREdkFQxFdPnXW2BcqlcrOtSEiIrIPhiLC/kudrHnqjIiIXBlDEV0ejs/bexARkQtjKCLl9Fk73t6DiIhcGEORi7tQUIIMYxEAoE0oQxEREbkuhiIXZ7m9R0SAJ3w93O1cGyIiIvthKHJxB5UrWbM/ERERubYahaK5c+eic+fO0Ov10Ov1iI2NxU8//aTMLyoqQlJSEho1agQfHx8MGDAAmZmZVss4ceIEEhIS4OXlhZCQEEyYMAFlZWVWZTZu3Iibb74ZOp0OrVq1wsKFCyvVZfbs2WjevDk8PDzQs2dPbN26tSabQpcc4JWsiYiIANQwFEVERGDGjBlITU3F9u3bcffdd+PBBx/E3r17AQDjxo3DihUrsHz5cvz66684c+YMHn74YeX1JpMJCQkJKCkpwZ9//onPP/8cCxcuxJQpU5Qy6enpSEhIwF133YW0tDSMHTsWI0aMwNq1a5UyS5cuxfjx4zF16lTs2LEDXbp0QXx8PLKysuraHi5nf4VrFBEREbk0qaOAgABZsGCB5Obmiru7uyxfvlyZt3//fgEgKSkpIiKyevVqUavVkpGRoZSZO3eu6PV6KS4uFhGRiRMnSocOHazWMXDgQImPj1ee9+jRQ5KSkpTnJpNJwsPDZfr06TWqu8FgEABiMBhq9DpnYTKZJfqVn6TZpJVyKNNo7+pUX36+CFD+yM+3d22IiKiB1dfvd637FJlMJixZsgQFBQWIjY1FamoqSktLERcXp5SJjo5G06ZNkZKSAgBISUlBp06dEBoaqpSJj4+H0WhUjjalpKRYLcNSxrKMkpISpKamWpVRq9WIi4tTylxNcXExjEaj1cOVnci5iMJSE7RuajRv5G3v6hAREdlVjUPR7t274ePjA51Oh5EjR+K7775D+/btkZGRAa1WC39/f6vyoaGhyMjIAABkZGRYBSLLfMu8a5UxGo0oLCxEdnY2TCZTlWUsy7ia6dOnw8/PT3lERkbWdPOdyoFLnazbhPrATcM+90RE5Npq/EvYtm1bpKWlYcuWLXj++eeRmJiIffv21UfdbG7y5MkwGAzK4+TJk/aukl1ZOlm3DeXIMyIiIreavkCr1aJVq1YAgJiYGGzbtg2zZs3CwIEDUVJSgtzcXKujRZmZmQgLCwMAhIWFVRolZhmdVrHMlSPWMjMzodfr4enpCY1GA41GU2UZyzKuRqfTQafT1XSTnZbl9h68kjUREZENrlNkNptRXFyMmJgYuLu7Izk5WZl38OBBnDhxArGxsQCA2NhY7N6922qU2Lp166DX69G+fXulTMVlWMpYlqHVahETE2NVxmw2Izk5WSlD1XMw89KRIg7HJyIiqtmRosmTJ6Nv375o2rQp8vLy8OWXX2Ljxo1Yu3Yt/Pz8MHz4cIwfPx6BgYHQ6/V44YUXEBsbi169egEA+vTpg/bt2+Opp57CO++8g4yMDLzyyitISkpSjuCMHDkSH3/8MSZOnIhhw4Zh/fr1WLZsGVatWqXUY/z48UhMTES3bt3Qo0cPzJw5EwUFBRg6dKgNm8a5FZaYcOx8AQCGIiIiIqCGoSgrKwtDhgzB2bNn4efnh86dO2Pt2rW49957AQAffPAB1Go1BgwYgOLiYsTHx2POnDnK6zUaDVauXInnn38esbGx8Pb2RmJiIl5//XWlTIsWLbBq1SqMGzcOs2bNQkREBBYsWID4+HilzMCBA3Hu3DlMmTIFGRkZ6Nq1K9asWVOp8zVd3eGsfIgAgd5aBPvwlCIREZFKRMTelbAXo9EIPz8/GAwG6PWu1dl4+faTmPD1LvRqGYglzzrYaceCAsDHp/zv/HzAm5cTICJyJfX1+81x2C7qoHJ7D9cKg0RERFfDUOSi2MmaiIjIGkORi7IcKWoTylBEREQEMBS5pAsFJcjKKwbAI0VEREQWDEUuyHLqLCLAEz66Gl+/k4iIyCkxFLmgy52seZSIiIjIgqHIBR3KKg9FrUIYioiIiCwYilzQkazyK1m3CvGxc02IiIhuHAxFLujwuXwADEVEREQVMRS5GENhKc5dGnkWFcwrQRMREVkwFLmYw1nlR4nC9B7w9XC3c22IiIhuHAxFLuZIFk+dERERVYWhyMWwPxEREVHVGIpcjOX0WRRDERERkRWGIhdz5NKRInayJiIissZQ5EKKSk04mXMRAE+fERERXYmhyIWkZxfALIDeww3BPjp7V4eIiOiGwlDkQg5XGHmmUqnsXBsiIqIbC0ORCznM4fhERERXxVDkQjgcn4iI6OoYilyI5cKNUcEMRURERFdiKHIRJrPgaHYBAB4pIiIiqgpDkYs4deEiSsrM0LqpERHgZe/qEBER3XAYilyEpZN1yyBvaNQceUZERHQlhiIXwZFnRERE18ZQ5CIYioiIiK6NochFcDg+ERHRtTEUuQARUY4UcTg+ERFR1RiKXMC5vGLkFZVBrQJaBHnbuzpEREQ3JIYiF2A5dRYZ6AUPd42da0NERHRjYihyAZYrWbfiqTMiIqKrYihyARx5RkREdH0MRS4g/fxFAOxPREREdC0MRS7g+Pnye541ZygiIiK6KoYiJ1dqMuPUhUIAQPNGDEVERERXw1Dk5E5fKITJLPBwVyPEV2fv6hAREd2wGIqc3LFLp86aBXpDzRvBEhERXRVDkZM7fqmTdbNGXnauCRER0Y2NocjJHWMnayIiomphKHJyJ3ikiIiIqFoYipyccqSII8+IiIiuiaHIiZnMgpM55cPxeaSIiIjo2hiKnNhZQyFKTGZoNWo09vO0d3WIiIhuaAxFTswy8iwy0BMaDscnIiK6JoYiJ3YixxKKeOqMiIjoehiKnNjZ3PL+RE38eeqMiIjoehiKnNjp3CIAQDhDERER0XUxFDmxs4byI0Xh/h52rgkREdGNj6HIiZ25dPqMI8+IiIiuj6HISYkIzhrKT5+xTxEREdH1MRQ5qZyCEhSXmaFSAaF6nj4jIiK6HoYiJ3XmUifrYB8dtG58m4mIiK6Hv5ZO6sylTtaNeeqMiIioWmoUiqZPn47u3bvD19cXISEh6N+/Pw4ePGhVpqioCElJSWjUqBF8fHwwYMAAZGZmWpU5ceIEEhIS4OXlhZCQEEyYMAFlZWVWZTZu3Iibb74ZOp0OrVq1wsKFCyvVZ/bs2WjevDk8PDzQs2dPbN26tSab49TOKNco4qkzIiKi6qhRKPr111+RlJSEzZs3Y926dSgtLUWfPn1QUFCglBk3bhxWrFiB5cuX49dff8WZM2fw8MMPK/NNJhMSEhJQUlKCP//8E59//jkWLlyIKVOmKGXS09ORkJCAu+66C2lpaRg7dixGjBiBtWvXKmWWLl2K8ePHY+rUqdixYwe6dOmC+Ph4ZGVl1aU9nIalkzVHnhEREVWT1EFWVpYAkF9//VVERHJzc8Xd3V2WL1+ulNm/f78AkJSUFBERWb16tajVasnIyFDKzJ07V/R6vRQXF4uIyMSJE6VDhw5W6xo4cKDEx8crz3v06CFJSUnKc5PJJOHh4TJ9+vRq199gMAgAMRgMNdhqx/DPxanSbNJKWfDbUXtXxfby80WA8kd+vr1rQ0REDay+fr/r1KfIYDAAAAIDAwEAqampKC0tRVxcnFImOjoaTZs2RUpKCgAgJSUFnTp1QmhoqFImPj4eRqMRe/fuVcpUXIaljGUZJSUlSE1NtSqjVqsRFxenlKlKcXExjEaj1cNZWW7xEe7H02dERETVUetQZDabMXbsWNx6663o2LEjACAjIwNarRb+/v5WZUNDQ5GRkaGUqRiILPMt865Vxmg0orCwENnZ2TCZTFWWsSyjKtOnT4efn5/yiIyMrPmGO4gzvMUHERFRjdQ6FCUlJWHPnj1YsmSJLetTryZPngyDwaA8Tp48ae8q1YtSkxlZeZf6FLGjNRERUbW41eZFo0aNwsqVK7Fp0yZEREQo08PCwlBSUoLc3Fyro0WZmZkICwtTylw5SswyOq1imStHrGVmZkKv18PT0xMajQYajabKMpZlVEWn00Gn09V8gx1MVl4xzAK4a1QI8nb+7SUiIrKFGh0pEhGMGjUK3333HdavX48WLVpYzY+JiYG7uzuSk5OVaQcPHsSJEycQGxsLAIiNjcXu3butRomtW7cOer0e7du3V8pUXIaljGUZWq0WMTExVmXMZjOSk5OVMq4s49I1ikL1HlCrVXauDRERkWOo0ZGipKQkfPnll/jhhx/g6+ur9N/x8/ODp6cn/Pz8MHz4cIwfPx6BgYHQ6/V44YUXEBsbi169egEA+vTpg/bt2+Opp57CO++8g4yMDLzyyitISkpSjuKMHDkSH3/8MSZOnIhhw4Zh/fr1WLZsGVatWqXUZfz48UhMTES3bt3Qo0cPzJw5EwUFBRg6dKit2sZhXR6Oz1NnRERE1VaToWoAqnx89tlnSpnCwkL55z//KQEBAeLl5SUPPfSQnD171mo5x44dk759+4qnp6cEBQXJv/71LyktLbUqs2HDBunatatotVpp2bKl1TosPvroI2natKlotVrp0aOHbN68uSab47RD8j/ddESaTVopo77cYe+q1A8OyScicmn19futEhGxXySzL6PRCD8/PxgMBuj1entXx2beXLkPC35PxzO3t8DLCe3tXR3bKygAfHzK/87PB7y97VsfIiJqUPX1+817nzmhs8by02dhvJo1ERFRtTEUOaEM9ikiIiKqMYYiJ2QJRWEMRURERNXGUORkzGZBpuX0mZ6hiIiIqLoYipxMdkExyswCtQoI9uWFG4mIiKqLocjJWE6dBfvq4K7h20tERFRd/NV0MmcNHHlGRERUGwxFTsbSn6gx+xMRERHVCEORk8kyFgMAQvTsT0RERFQTDEVO5lxeeSgK8mEoIiIiqgmGIieTnc9QREREVBsMRU7mcijS2rkmREREjoWhyMlk55cAAIJ4jSIiIqIaYShyIiKCc5eOFAXz9BkREVGNMBQ5kbziMpSUmQGwTxEREVFNMRQ5kexLI898dG7w1GrsXBsiIiLHwlDkRJT+ROxkTUREVGMMRU6E1ygiIiKqPYYiJ8JrFBEREdUeQ5ETUUKRL0+fERER1RRDkRPhkSIiIqLaYyhyIufyLB2tGYqIiIhqiqHIifBIERERUe0xFDkRSygK5i0+iIiIaoyhyEmIyOVQxCNFRERENcZQ5CQKSkwoKr10iw+OPiMiIqoxhiInYblwo5dWAy+tm51rQ0RE5HgYipwEO1kTERHVDUORk8hWbvHBU2dERES1wVDkJHikiIiIqG4YipzEufxLF27kcHwiIqJaYShyEhyOT0REVDcMRU5C6VPEI0VERES1wlDkJC4fKWJHayIiotpgKHIS2fm8GSwREVFdMBQ5iXN5HH1GRERUFwxFTqCguAyFpSYA7FNERERUWwxFTsDSn8jDXQ1vrcbOtSEiInJMDEVOoOKFG1UqlZ1rQ0RE5JgYipzAubzyTtbBPHVGRERUawxFToC3+CAiIqo7hiInwFBERERUdwxFToAXbiQiIqo7hiIncI63+CAiIqozhiInwKtZExER1R1DkRNgnyIiIqK6YyhyAtnKLT7Yp4iIiKi2GIocXGGJCQUlvMUHERFRXTEUOTjLqTOdmxq+Ojc714aIiMhxMRQ5uHO8xQcREZFNMBQ5uGwOxyciIrIJhiIHZxmOzws3EhER1Q1DkYNTLtzI4fhERER1wlDk4HiNIiIiItuocSjatGkT7r//foSHh0OlUuH777+3mi8imDJlCho3bgxPT0/ExcXh0KFDVmVycnIwePBg6PV6+Pv7Y/jw4cjPz7cqs2vXLtx+++3w8PBAZGQk3nnnnUp1Wb58OaKjo+Hh4YFOnTph9erVNd0ch5eVVwQACNEzFBEREdVFjUNRQUEBunTpgtmzZ1c5/5133sGHH36IefPmYcuWLfD29kZ8fDyKioqUMoMHD8bevXuxbt06rFy5Eps2bcKzzz6rzDcajejTpw+aNWuG1NRUvPvuu3jttdcwf/58pcyff/6JQYMGYfjw4di5cyf69++P/v37Y8+ePTXdJIeWaSw/UhTi62HnmhARETk4qQMA8t133ynPzWazhIWFybvvvqtMy83NFZ1OJ1999ZWIiOzbt08AyLZt25QyP/30k6hUKjl9+rSIiMyZM0cCAgKkuLhYKTNp0iRp27at8vyxxx6ThIQEq/r07NlTnnvuuWrX32AwCAAxGAzVfs2NJvbtX6TZpJWy88QFe1el4eTniwDlj/x8e9eGiIgaWH39ftu0T1F6ejoyMjIQFxenTPPz80PPnj2RkpICAEhJSYG/vz+6deumlImLi4NarcaWLVuUMnfccQe02ssjquLj43Hw4EFcuHBBKVNxPZYylvVUpbi4GEaj0erhyMxmQdaljtahPH1GRERUJzYNRRkZGQCA0NBQq+mhoaHKvIyMDISEhFjNd3NzQ2BgoFWZqpZRcR1XK2OZX5Xp06fDz89PeURGRtZ0E28oORdLUGYWqFTsaE1ERFRXLjX6bPLkyTAYDMrj5MmT9q5SnWQay/tpNfLWwV3jUm8lERGRzdn0lzQsLAwAkJmZaTU9MzNTmRcWFoasrCyr+WVlZcjJybEqU9UyKq7jamUs86ui0+mg1+utHo4sy8hTZ0RERLZi01DUokULhIWFITk5WZlmNBqxZcsWxMbGAgBiY2ORm5uL1NRUpcz69ethNpvRs2dPpcymTZtQWlqqlFm3bh3atm2LgIAApUzF9VjKWNbjCixHikL1HHlGRERUVzUORfn5+UhLS0NaWhqA8s7VaWlpOHHiBFQqFcaOHYs333wTP/74I3bv3o0hQ4YgPDwc/fv3BwC0a9cO9913H5555hls3boVf/zxB0aNGoXHH38c4eHhAIAnnngCWq0Ww4cPx969e7F06VLMmjUL48ePV+oxZswYrFmzBu+99x4OHDiA1157Ddu3b8eoUaPq3ioOIpNHioiIiGynpsPVNmzYIAAqPRITE0WkfFj+q6++KqGhoaLT6eSee+6RgwcPWi3j/PnzMmjQIPHx8RG9Xi9Dhw6VvLw8qzJ//fWX3HbbbaLT6aRJkyYyY8aMSnVZtmyZtGnTRrRarXTo0EFWrVpVo21x9CH5k7/dJc0mrZT3fz54/cLOhEPyiYhcWn39fqtEROyYyezKaDTCz88PBoPBIfsXjfh8G37Zn4W3H+qEJ3o2tXd1Gk5BAeDjU/53fj7g7W3f+hARUYOqr99vDllyYDx9RkREZDsMRQ6MHa2JiIhsh6HIQZWZzMjOv3TfMx4pIiIiqjOGIgd1vqAEZgE0ahUaeTMUERER1RVDkYOynDoL9tFBo1bZuTZERESOj6HIQbGTNRERkW0xFDkoy5GiEHayJiIisgmGIgeVpYw845EiIiIiW2AoclAZllDkyyNFREREtsBQ5KAu9yliKCIiIrIFhiIHdblPEU+fERER2QJDkYPKyis/UhTmxyNFREREtsBQ5ICKy0zIKSgBAISwTxEREZFNMBQ5oDO55afOPN01CPByt3NtiIiInANDkQM6faEQANAkwBMqFa9mTUREZAsMRQ7o1IWLAICIAE8714SIiMh5MBQ5oNO5l44U+TMUERER2QpDkQM6den0WUSAl51rQkRE5DwYihxQxT5FREREZBsMRQ6IfYqIiIhsj6HIwZSazMp9zxiKiIiIbIehyMFkGIpgFkDrpkaQN2/xQUREZCsMRQ7mpOXUmb8n1Gpeo4iIiMhWGIocDDtZExER1Q+GIgdzeTg+QxEREZEtMRQ5GF64kYiIqH4wFDmYy8PxeeFGIiIiW2IocjCn2KeIiIioXjAUOZAykxkZBl6jiIiIqD4wFDmQs4YilJkF7hoVQnw97F0dIiIip8JQ5ECOnS8AADQN9IKG1ygiIiKyKYYiB3IsuzwUtQjytnNNiIiInA9DkQM5ylBERERUbxiKHIjlSFFzhiIiIiKbYyhyIOk8UkRERFRvGIocRKnJjJOXrlHEUERERGR7DEUO4tSFQpjMAk93DUI5HJ+IiMjmGIocRHp2PgCgWSMvqDkcn4iIyOYYihzE0XPl/YlaBvPUGRERUX1gKHIQR86VHymKCvaxc02IiIicE0ORg/g7szwUtQ71tXNNiIiInBNDkQMQEfydmQcAaB3CI0VERET1gaHIAWQai5FXVAaNWsU+RURERPWEocgBWI4SNWvkBZ2bxs61ISIick4MRQ7AEorahLA/ERERUX1hKHIAe88YAQDRjRmKiIiI6gtDkQNIO5kLAOga6W/XehARETkzhqIbXO7FEuVGsAxFRERE9Yeh6Ab31ykDAKB5Iy/4e2ntXBsiIiLnxVB0g0s7kQuAR4mIiIjqG0PRDe6vU7kAGIqIiIjqG0PRDaykzIztx3IAAF0YioiIiOoVQ9EN7I/D2TAWlSHYV4fOEf72rg4REZFTYyi6ga3YdQYA0K9jGDRqlZ1rQ0RE5NwcPhTNnj0bzZs3h4eHB3r27ImtW7fau0o2YSgsxdo9GQCAhM7hdq4NERGR83PoULR06VKMHz8eU6dOxY4dO9ClSxfEx8cjKyvL3lWrs083HUVBiQnRYb7o3jzA3tUhIiJyeioREXtXorZ69uyJ7t274+OPPwYAmM1mREZG4oUXXsCLL75YqXxxcTGKi4uV50ajEZGRkTAYDNDr9Tar1/s/H4SxqAwiAgEgAgjk0r/lz2F5XmGe+dLfFwpKsOHgOQDA7CduRkLnxjarm1MoKAB8fMr/zs8HvL3tWx8iImpQRqMRfn5+Nv/9drPZkhpYSUkJUlNTMXnyZGWaWq1GXFwcUlJSqnzN9OnTMW3atHqv21fbTuJcXvH1C17Hc71bol+nMBvUiIiIiK7HYUNRdnY2TCYTQkNDraaHhobiwIEDVb5m8uTJGD9+vPLccqTI1obe2hwXi01QqQAVAKhUUJX/AxVUynSVClCpyjtQq1WXp2vUKvRq2Qgdm/jZvG5ERERUNYcNRbWh0+mg0+nqfT3/vLNVva+DiIiIbMthO1oHBQVBo9EgMzPTanpmZibCwnjKiYiIiGrGYUORVqtFTEwMkpOTlWlmsxnJycmIjY21Y82IiIjIETn06bPx48cjMTER3bp1Q48ePTBz5kwUFBRg6NCh9q4aERERORiHDkUDBw7EuXPnMGXKFGRkZKBr165Ys2ZNpc7XRERERNfj0Ncpqqv6us4B1TNep4iIyKXV1++3w/YpIiIiIrIlhiIiIiIiMBQRERERAWAoIiIiIgLAUEREREQEgKGIiIiICABDEREREREAhiIiIiIiAA5+Reu6sly30mg02rkmVCMFBZf/NhoBk8l+dSEiogZn+d229fWnXToU5eXlAQAiIyPtXBOqtfBwe9eAiIjsJC8vD35+fjZbnkvf5sNsNuPMmTPw9fWFSqWy2XKNRiMiIyNx8uRJ3j7kErZJZWyTytgmVWO7VMY2qcyV2kREkJeXh/DwcKjVtusJ5NJHitRqNSIiIupt+Xq93uk/mDXFNqmMbVIZ26RqbJfK2CaVuUqb2PIIkQU7WhMRERGBoYiIiIgIAENRvdDpdJg6dSp0Op29q3LDYJtUxjapjG1SNbZLZWyTytgmdefSHa2JiIiILHikiIiIiAgMRUREREQAGIqIiIiIADAUEREREQFgKCIiIiICwFBUL2bPno3mzZvDw8MDPXv2xNatW+1dpXoxffp0dO/eHb6+vggJCUH//v1x8OBBqzJFRUVISkpCo0aN4OPjgwEDBiAzM9OqzIkTJ5CQkAAvLy+EhIRgwoQJKCsra8hNqTczZsyASqXC2LFjlWmu2CanT5/Gk08+iUaNGsHT0xOdOnXC9u3blfkigilTpqBx48bw9PREXFwcDh06ZLWMnJwcDB48GHq9Hv7+/hg+fDjy8/MbelNswmQy4dVXX0WLFi3g6emJqKgovPHGG1Y3t3SFNtm0aRPuv/9+hIeHQ6VS4fvvv7eab6s22LVrF26//XZ4eHggMjIS77zzTn1vWq1dq01KS0sxadIkdOrUCd7e3ggPD8eQIUNw5swZq2U4W5s0KCGbWrJkiWi1Wvm///s/2bt3rzzzzDPi7+8vmZmZ9q6azcXHx8tnn30me/bskbS0NOnXr580bdpU8vPzlTIjR46UyMhISU5Olu3bt0uvXr3klltuUeaXlZVJx44dJS4uTnbu3CmrV6+WoKAgmTx5sj02yaa2bt0qzZs3l86dO8uYMWOU6a7WJjk5OdKsWTN5+umnZcuWLXL06FFZu3atHD58WCkzY8YM8fPzk++//17++usveeCBB6RFixZSWFiolLnvvvukS5cusnnzZvntt9+kVatWMmjQIHtsUp299dZb0qhRI1m5cqWkp6fL8uXLxcfHR2bNmqWUcYU2Wb16tbz88svy7bffCgD57rvvrObbog0MBoOEhobK4MGDZc+ePfLVV1+Jp6enfPLJJw21mTVyrTbJzc2VuLg4Wbp0qRw4cEBSUlKkR48eEhMTY7UMZ2uThsRQZGM9evSQpKQk5bnJZJLw8HCZPn26HWvVMLKysgSA/PrrryJSvgO7u7vL8uXLlTL79+8XAJKSkiIi5V8AarVaMjIylDJz584VvV4vxcXFDbsBNpSXlyetW7eWdevWSe/evZVQ5IptMmnSJLntttuuOt9sNktYWJi8++67yrTc3FzR6XTy1VdfiYjIvn37BIBs27ZNKfPTTz+JSqWS06dP11/l60lCQoIMGzbMatrDDz8sgwcPFhHXbJMrA4Ct2mDOnDkSEBBgte9MmjRJ2rZtW89bVHdVBcUrbd26VQDI8ePHRcT526S+8fSZDZWUlCA1NRVxcXHKNLVajbi4OKSkpNixZg3DYDAAAAIDAwEAqampKC0ttWqP6OhoNG3aVGmPlJQUdOrUCaGhoUqZ+Ph4GI1G7N27twFrb1tJSUlISEiw2nbANdvkxx9/RLdu3fDoo48iJCQEN910Ez799FNlfnp6OjIyMqzaxM/PDz179rRqE39/f3Tr1k0pExcXB7VajS1btjTcxtjILbfcguTkZPz9998AgL/++gu///47+vbtC8A12+RKtmqDlJQU3HHHHdBqtUqZ+Ph4HDx4EBcuXGigrak/BoMBKpUK/v7+ANgmdeVm7wo4k+zsbJhMJqsfMwAIDQ3FgQMH7FSrhmE2mzF27Fjceuut6NixIwAgIyMDWq1W2VktQkNDkZGRoZSpqr0s8xzRkiVLsGPHDmzbtq3SPFdsk6NHj2Lu3LkYP348XnrpJWzbtg2jR4+GVqtFYmKisk1VbXPFNgkJCbGa7+bmhsDAQIdskxdffBFGoxHR0dHQaDQwmUx46623MHjwYABwyTa5kq3aICMjAy1atKi0DMu8gICAeql/QygqKsKkSZMwaNAg6PV6AGyTumIoIptISkrCnj178Pvvv9u7KnZ18uRJjBkzBuvWrYOHh4e9q3NDMJvN6NatG95++20AwE033YQ9e/Zg3rx5SExMtHPt7GPZsmVYvHgxvvzyS3To0AFpaWkYO3YswsPDXbZNqGZKS0vx2GOPQUQwd+5ce1fHafD0mQ0FBQVBo9FUGkmUmZmJsLAwO9Wq/o0aNQorV67Ehg0bEBERoUwPCwtDSUkJcnNzrcpXbI+wsLAq28syz9GkpqYiKysLN998M9zc3ODm5oZff/0VH374Idzc3BAaGupybdK4cWO0b9/ealq7du1w4sQJAJe36Vr7TVhYGLKysqzml5WVIScnxyHbZMKECXjxxRfx+OOPo1OnTnjqqacwbtw4TJ8+HYBrtsmVbNUGzrY/AZcD0fHjx7Fu3TrlKBHgum1iKwxFNqTVahETE4Pk5GRlmtlsRnJyMmJjY+1Ys/ohIhg1ahS+++47rF+/vtLh2JiYGLi7u1u1x8GDB3HixAmlPWJjY7F7926rndiyk1/5Q+oI7rnnHuzevRtpaWnKo1u3bhg8eLDyt6u1ya233lrpUg1///03mjVrBgBo0aIFwsLCrNrEaDRiy5YtVm2Sm5uL1NRUpcz69ethNpvRs2fPBtgK27p48SLUauuvX41GA7PZDMA12+RKtmqD2NhYbNq0CaWlpUqZdevWoW3btg55msgSiA4dOoRffvkFjRo1sprvim1iU/bu6e1slixZIjqdThYuXCj79u2TZ599Vvz9/a1GEjmL559/Xvz8/GTjxo1y9uxZ5XHx4kWlzMiRI6Vp06ayfv162b59u8TGxkpsbKwy3zL8vE+fPpKWliZr1qyR4OBghx1+XpWKo89EXK9Ntm7dKm5ubvLWW2/JoUOHZPHixeLl5SVffPGFUmbGjBni7+8vP/zwg+zatUsefPDBKode33TTTbJlyxb5/fffpXXr1g41/LyixMREadKkiTIk/9tvv5WgoCCZOHGiUsYV2iQvL0927twpO3fuFADy/vvvy86dO5WRVLZog9zcXAkNDZWnnnpK9uzZI0uWLBEvL68bdvj5tdqkpKREHnjgAYmIiJC0tDSr792KI8mcrU0aEkNRPfjoo4+kadOmotVqpUePHrJ582Z7V6leAKjy8dlnnyllCgsL5Z///KcEBASIl5eXPPTQQ3L27Fmr5Rw7dkz69u0rnp6eEhQUJP/617+ktLS0gbem/lwZilyxTVasWCEdO3YUnU4n0dHRMn/+fKv5ZrNZXn31VQkNDRWdTif33HOPHDx40KrM+fPnZdCgQeLj4yN6vV6GDh0qeXl5DbkZNmM0GmXMmDHStGlT8fDwkJYtW8rLL79s9cPmCm2yYcOGKr9DEhMTRcR2bfDXX3/JbbfdJjqdTpo0aSIzZsxoqE2ssWu1SXp6+lW/dzds2KAsw9napCGpRCpcQpWIiIjIRbFPEREREREYioiIiIgAMBQRERERAWAoIiIiIgLAUEREREQEgKGIiIiICABDEREREREAhiIiIiIiAAxFRERERAAYioiIiIgAMBQRERERAQD+P70xpn+pwq6HAAAAAElFTkSuQmCC",
115
+ "text/plain": [
116
+ "<Figure size 640x480 with 1 Axes>"
117
+ ]
118
+ },
119
+ "metadata": {},
120
+ "output_type": "display_data"
121
+ }
122
+ ],
123
+ "source": [
124
+ "plt.plot([len([l for l in lens if l <= m]) for m in range(max(lens) + 1)])\n",
125
+ "plt.title(\"Number of fully covered examples as a function of max length\")\n",
126
+ "plt.axvline(x=256, color=\"red\")"
127
+ ]
128
+ },
129
+ {
130
+ "attachments": {},
131
+ "cell_type": "markdown",
132
+ "metadata": {},
133
+ "source": [
134
+ "Percentage of tokens left out:"
135
+ ]
136
+ },
137
+ {
138
+ "cell_type": "code",
139
+ "execution_count": 4,
140
+ "metadata": {},
141
+ "outputs": [
142
+ {
143
+ "data": {
144
+ "text/plain": [
145
+ "<matplotlib.lines.Line2D at 0x7f6eef392020>"
146
+ ]
147
+ },
148
+ "execution_count": 4,
149
+ "metadata": {},
150
+ "output_type": "execute_result"
151
+ },
152
+ {
153
+ "data": {
154
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGzCAYAAABzfl4TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABHN0lEQVR4nO3deVxU9f4/8NfMwAz7sIMoAqJprplb5q6UmllWWnnN1G7eb2WlZZv1q7Qytc26btmmZaVmN225mbt5K/dyV8IVXFGRHQaYef/+gBkZBpCBmTkw83o+nIfMmbO8z4eBeXHO53yOSkQERERERA6gVroAIiIich8MFkREROQwDBZERETkMAwWRERE5DAMFkREROQwDBZERETkMAwWRERE5DAMFkREROQwDBZERETkMAwWbkKlUuHxxx9XugzyICUlJXjuuecQGxsLtVqNYcOGKV1SpRYvXgyVSoWTJ08qXUq13n77bTRr1gwajQY33HCD0uXUycmTJ6FSqbB48WKlS7mm+Ph43H777UqX4VYYLBSkUqlq9Ni8ebPSpRLZ+Oyzz/D2229j+PDh+Pzzz/HUU08pWs+bb76JVatWKVpDba1duxbPPfccevTogUWLFuHNN99UuiS3cujQIUydOrXeh0t34aV0AZ5syZIlVs+/+OILrFu3zmb69ddf78qyiGpk48aNaNy4MWbPnq10KQBKg8Xw4cNtjpyMHj0a999/P3Q6nTKF1cDGjRuhVqvx6aefQqvVKl2O2zl06BCmTZuGvn37Ij4+Xuly3B6DhYIeeOABq+fbtm3DunXrbKaTc+Xl5cHf31/pMhqc9PR0BAcHK13GNWk0Gmg0GqXLqFZ6ejp8fX0ZKsgt8FRIPZeXl4fJkycjNjYWOp0OLVu2xDvvvIOa3JT2jTfegFqtxpw5cyzTVq9ejV69esHf3x+BgYEYMmQIDh48aLXc2LFjERAQgDNnzmDYsGEICAhAREQEnnnmGRiNxhrVvXr1avTp0weBgYEICgpCly5d8PXXX1vNs2LFCnTq1Am+vr4IDw/HAw88gDNnzlhef+edd6BSqXDq1Cmb9U+ZMgVarRZXrlyxTNu+fTsGDRoEvV4PPz8/9OnTB7///rvVclOnToVKpcKhQ4fwj3/8AyEhIejZsycAYN++fRg7diyaNWsGHx8fREdH46GHHsLly5dttr9582Z07twZPj4+SExMxMKFCy3rrujLL7+07GdoaCjuv/9+pKWlXbMNT506hcceewwtW7aEr68vwsLCMGLECJvDucXFxZg2bRpatGgBHx8fhIWFoWfPnli3bl2168/IyMAzzzyDdu3aISAgAEFBQRg8eDD27t1b7XLm8+ebNm3CwYMHrU7Zbd68udLTd5Wdc7fnfWYymfDBBx+gXbt28PHxQUREBAYNGoRdu3YBKD2tmJeXh88//9xSz9ixYwFU3cdi/vz5aNOmDXQ6HWJiYjBhwgRkZmZazdO3b1+0bdsWhw4dQr9+/eDn54fGjRvjrbfeqraNzEpKSvD6668jMTEROp0O8fHxePHFF2EwGCzzqFQqLFq0CHl5eZbaq+ubYK5p37596NOnD/z8/NC8eXN8++23AIBff/0V3bp1g6+vL1q2bIn169dbLV+T95WIoF+/foiIiEB6erplelFREdq1a4fExETk5eXVqA3KO3LkCIYPH47Q0FD4+Pigc+fO+OGHH6zmMX+/fv/9dzz99NOIiIiAv78/7rrrLly8eNFqXpPJhKlTpyImJgZ+fn7o168fDh06hPj4eKvv/4gRIwAA/fr1q/IU82+//YauXbvCx8cHzZo1wxdffGH3/lEZoXpjwoQJUv5bYjKZpH///qJSqeThhx+WuXPnytChQwWATJo0yWpZADJhwgTL85deeklUKpV89NFHlmlffPGFqFQqGTRokMyZM0dmzZol8fHxEhwcLCdOnLDMN2bMGPHx8ZE2bdrIQw89JAsWLJB77rlHAMj8+fOvuR+LFi0SlUolbdu2lenTp8u8efPk4YcfltGjR1vNA0C6dOkis2fPlhdeeEF8fX0lPj5erly5IiIip06dEpVKJW+99ZbNNpo1ayZDhgyxPN+wYYNotVrp3r27vPvuuzJ79mxp3769aLVa2b59u2W+V199VQBI69at5c4775T58+fLvHnzRETknXfekV69eslrr70mH330kUycOFF8fX2la9euYjKZLOv4888/RafTSXx8vMycOVOmT58uMTEx0qFDB6n4I/XGG2+ISqWS++67T+bPny/Tpk2T8PBwq/2syooVK6RDhw7yyiuvyEcffSQvvviihISESFxcnOTl5Vnme/HFF0WlUsn48ePl448/lnfffVdGjhwpM2fOrHb9O3fulMTERHnhhRdk4cKF8tprr0njxo1Fr9fLmTNnqlwuNzdXlixZIq1atZImTZrIkiVLZMmSJXL+/HnZtGmTAJBNmzZZLXPixAkBIIsWLbJMs+d9NnbsWAEggwcPlvfff1/eeecdufPOO2XOnDkiIrJkyRLR6XTSq1cvSz1//PGHiFx9r5V/j5vfB0lJSTJnzhx5/PHHRaPRSJcuXaSoqMgyX58+fSQmJkZiY2Nl4sSJMn/+fOnfv78AkJ9//rna9jXvIwAZPny4zJs3Tx588EEBIMOGDbPMs2TJEunVq5fodDpL7ceOHatyneVrevbZZ2XOnDnSunVr0Wg0smzZMomOjpapU6fK+++/b/l+ZmdnW5av6fvq+PHjEhAQIHfddZdl2gsvvCAqlUp+/fXXave7su/3gQMHRK/XS+vWrWXWrFkyd+5c6d27t6hUKvnuu+8s85m/Xx07dpT+/fvLnDlzZPLkyaLRaOTee++12s5zzz0nAGTo0KEyd+5cGT9+vDRp0kTCw8NlzJgxIiJy7NgxefLJJwWAvPjii1bvVxGRuLg4admypURFRcmLL74oc+fOlRtvvFFUKpUcOHCg2v2kyjFY1CMVg8WqVasEgLzxxhtW8w0fPlxUKpUcPXrUMq18sJg8ebKo1WpZvHix5fWcnBwJDg6W8ePHW63r/Pnzotfrraabfxm+9tprVvN27NhROnXqVO0+ZGZmSmBgoHTr1k0KCgqsXjN/OBcVFUlkZKS0bdvWap6ffvpJAMgrr7ximda9e3ebbe7YsUMAyBdffGFZb4sWLWTgwIFWASA/P18SEhLklltusUwzf6CMHDnSpvb8/HybaUuXLhUAsmXLFsu0oUOHip+fn9WHb0pKinh5eVl9/06ePCkajUamT59utc79+/eLl5eXzfSa1LN161arfRcR6dChg1XIqqnCwkIxGo1W006cOCE6nc7me1+ZPn36SJs2baym2RssavI+27hxowCQJ5980qaG8t9vf39/y4dJeRWDRXp6umi1Wrn11lut9n/u3LkCQD777DOrfazY3gaDQaKjo+Wee+6xbZRy9uzZIwDk4Ycftpr+zDPPCADZuHGjVVv4+/tXu76KNX399deWaUeOHBEAolarZdu2bZbpa9assWn3mr6vREQWLlwoAOTLL7+Ubdu2iUajsfmjpjKVfb8HDBgg7dq1k8LCQss0k8kkN998s7Ro0cIyzfz9SkpKsvr+PvXUU6LRaCQzM1NESn93eXl5WYU0EZGpU6cKAKv3wooVKyp9X4qUBouKP+Pp6emi0+lk8uTJ19xXssVTIfXYzz//DI1GgyeffNJq+uTJkyEiWL16tdV0EcHjjz+ODz74AF9++SXGjBljeW3dunXIzMzEyJEjcenSJctDo9GgW7du2LRpk832H3nkEavnvXr1wvHjx6uted26dcjJycELL7wAHx8fq9fMpwl27dqF9PR0PPbYY1bzDBkyBK1atcJ///tfy7T77rsPu3fvxrFjxyzTli9fDp1OhzvvvBMAsGfPHqSkpOAf//gHLl++bNm3vLw8DBgwAFu2bIHJZKp23wDA19fX8nVhYSEuXbqEm266CQDw559/AgCMRiPWr1+PYcOGISYmxjJ/8+bNMXjwYKv1fffddzCZTLj33nut2jw6OhotWrSotM2rqqe4uBiXL19G8+bNERwcbKkHAIKDg3Hw4EGkpKRUu76KdDod1Gq1Zb8uX76MgIAAtGzZ0mr9znat99l//vMfqFQqvPrqqzbLVnbq6VrWr1+PoqIiTJo0ybL/ADB+/HgEBQVZvf8AICAgwKrfk1arRdeuXa/5s/Dzzz8DAJ5++mmr6ZMnTwYAm+3YIyAgAPfff7/lecuWLREcHIzrr78e3bp1s0w3f12+1pq+rwDgX//6FwYOHIgnnngCo0ePRmJiYq2uWMnIyMDGjRtx7733Iicnx/KzcPnyZQwcOBApKSlWp0HN2y7//e3VqxeMRqPl1OiGDRtQUlKCxx57zGq5J554wu76WrdujV69elmeR0REoGXLltf8HlPlFAsWW7ZswdChQxETEwOVSlWry8REBO+88w6uu+466HQ6NG7cGNOnT3d8sQo5deoUYmJiEBgYaDXdfJVIxb4HX3zxBebNm4c5c+Zg5MiRVq+ZP3T69++PiIgIq8fatWutzqMCsJzHLi8kJMSqT0NlzAGgbdu21e4XUPrLsKJWrVpZ7deIESOgVquxfPlyAKXf8xUrVmDw4MEICgqy2rcxY8bY7Nsnn3wCg8GArKwsq+0kJCTYbDsjIwMTJ05EVFQUfH19ERERYZnPvHx6ejoKCgrQvHlzm+UrTktJSYGIoEWLFjZ1HT582KbNKyooKMArr7xi6V8THh6OiIgIZGZmWu3Pa6+9hszMTFx33XVo164dnn32Wezbt6/adQOl56dnz56NFi1aWK1/3759Nu3lLDV5nx07dgwxMTEIDQ11yDarev9ptVo0a9bM5ueqSZMmNgGmJj8Lp06dglqttnlfREdHIzg4uNK+QzVVWU16vR6xsbE20wBY1VrT95XZp59+ivz8fKSkpGDx4sVWwaSmjh49ChHByy+/bPOzYA6MFX8emjZtavU8JCTEal/M7VexfUNDQy3z1lTFbZm3d63vMVVOsatC8vLy0KFDBzz00EO4++67a7WOiRMnYu3atXjnnXfQrl07ZGRkICMjw8GVNhw9evTAnj17MHfuXNx7771Wv4jNf7EvWbIE0dHRNst6eVm/FepLL/qYmBj06tUL33zzDV588UVs27YNqampmDVrlmUe8769/fbbVQ4sFBAQYPW8sl+O9957L/744w88++yzuOGGGxAQEACTyYRBgwbZHPGoCZPJBJVKhdWrV1fanhVrquiJJ57AokWLMGnSJHTv3h16vR4qlQr333+/VT29e/fGsWPH8P3332Pt2rX45JNPMHv2bHz44Yd4+OGHq1z/m2++iZdffhkPPfQQXn/9dYSGhkKtVmPSpEm12l+g6iMIVXX6rS/vs+pUVaPUoAM1ULujKtdSVU01qbWm7yuzzZs3Wzqb7t+/H927d7e7XvN6n3nmGQwcOLDSeSoGhLq2uz1cuS1PoFiwGDx4sM2h4/IMBgNeeuklLF26FJmZmWjbti1mzZqFvn37AgAOHz6MBQsW4MCBA5a/PCr7K7Qhi4uLw/r165GTk2N11OLIkSOW18tr3rw53nrrLfTt2xeDBg3Chg0bLMslJiYCACIjI5GUlOS0ms3bOXDgQKV/1ZevOzk5Gf3797d6LTk52Wa/7rvvPjz22GNITk7G8uXL4efnh6FDh9psMygoqNb7duXKFWzYsAHTpk3DK6+8Yple8fRCZGQkfHx8cPToUZt1VJyWmJgIEUFCQgKuu+46u2v69ttvMWbMGLz77ruWaYWFhTZXLgClf6WNGzcO48aNQ25uLnr37o2pU6dWGyy+/fZb9OvXD59++qnV9MzMTISHh9tdL3D1r8qKNdblr/PExESsWbMGGRkZ1R61qOkHePn3X7NmzSzTi4qKcOLECYf9fMTFxcFkMiElJcVqLJoLFy4gMzPT5n3uKva8r86dO4cnnngCt956K7RarSUY2Fu7uZ29vb0d2r5A6c9d+d/9ly9ftjnS4IxwR1Wrt30sHn/8cWzduhXLli3Dvn37MGLECAwaNMjyi/7HH39Es2bN8NNPPyEhIQHx8fF4+OGH3eqIxW233Qaj0Yi5c+daTZ89ezZUKlWlwax9+/b4+eefcfjwYQwdOhQFBQUAgIEDByIoKAhvvvkmiouLbZareBlXbd16660IDAzEjBkzUFhYaPWaOf137twZkZGR+PDDD60uu1u9ejUOHz6MIUOGWC13zz33QKPRYOnSpVixYgVuv/12q3EnOnXqhMTERLzzzjvIzc2t1b6Z/2Kp+BfK+++/bzNfUlISVq1ahbNnz1qmHz161KbPy9133w2NRoNp06bZrFdEKr2MteK2Ki43Z84cm7/+K64nICAAzZs3t2rbmq5/xYoVNue67REXFweNRoMtW7ZYTZ8/f36t13nPPfdARDBt2jSb18rX7+/vX+mHY0VJSUnQarX497//bbX8p59+iqysLJv3X23ddtttAGzfQ++99x4AOGw79qrp+woo7XdiMpnw6aef4qOPPoKXlxf++c9/2v2XfGRkJPr27YuFCxfi3LlzNq/X5vfPgAED4OXlhQULFlhNr/j7EoDl90VN3h9Ud/VygKzU1FQsWrQIqamplg5yzzzzDH755RfLcLfHjx/HqVOnsGLFCnzxxRcwGo146qmnMHz4cGzcuFHhPXCMoUOHol+/fnjppZdw8uRJdOjQAWvXrsX333+PSZMmWf5Sr+imm27C999/j9tuuw3Dhw/HqlWrEBQUhAULFmD06NG48cYbcf/99yMiIgKpqan473//ix49elT6A2mvoKAgzJ49Gw8//DC6dOliGSti7969yM/Px+effw5vb2/MmjUL48aNQ58+fTBy5EhcuHABH3zwAeLj422Gho6MjES/fv3w3nvvIScnB/fdd5/V62q1Gp988gkGDx6MNm3aYNy4cWjcuDHOnDmDTZs2ISgoCD/++OM16+7duzfeeustFBcXo3Hjxli7di1OnDhhM+/UqVOxdu1a9OjRA48++qgl/LVt2xZ79uyxzJeYmIg33ngDU6ZMwcmTJzFs2DAEBgbixIkTWLlyJf71r3/hmWeeqbKm22+/HUuWLIFer0fr1q2xdetWrF+/HmFhYVbztW7dGn379kWnTp0QGhqKXbt24dtvv73mvWNuv/12vPbaaxg3bhxuvvlm7N+/H1999ZXVX/H20uv1GDFiBObMmQOVSoXExET89NNP1+xPUp1+/fph9OjR+Pe//42UlBTLqan//e9/6Nevn2U/O3XqhPXr1+O9995DTEwMEhISrDoymkVERGDKlCmYNm0aBg0ahDvuuAPJycmYP38+unTp4rAB6jp06IAxY8bgo48+QmZmJvr06YMdO3bg888/x7Bhw9CvXz+HbMdeNX1fLVq0CP/973+xePFiNGnSBEBpAHnggQewYMECm06T1zJv3jz07NkT7dq1w/jx49GsWTNcuHABW7duxenTp685fkpFUVFRmDhxIt59913ccccdGDRoEPbu3YvVq1cjPDzc6ijFDTfcAI1Gg1mzZiErKws6nQ79+/dHZGSkXdukGnLlJShVASArV660PDdfdujv72/18PLyslzHPH78eAEgycnJluV2794tAOTIkSOu3gWHqHi5qUjpZaJPPfWUxMTEiLe3t7Ro0ULefvttq8uwRGzHsRAR+f7778XLy0vuu+8+y2V1mzZtkoEDB4perxcfHx9JTEyUsWPHyq5duyzLVXXpm/lSzZr44Ycf5OabbxZfX18JCgqSrl27ytKlS63mWb58uXTs2FF0Op2EhobKqFGj5PTp05Wu7+OPPxYAEhgYaHMZq9lff/0ld999t4SFhYlOp5O4uDi59957ZcOGDTb7cPHiRZvlT58+LXfddZcEBweLXq+XESNGyNmzZwWAvPrqq1bzbtiwQTp27CharVYSExPlk08+kcmTJ4uPj4/Nev/zn/9Iz549Le/jVq1ayYQJE6zeu5W5cuWKjBs3TsLDwyUgIEAGDhwoR44ckbi4OKtL6d544w3p2rWrBAcHi6+vr7Rq1UqmT59uNR5DZQoLC2Xy5MnSqFEj8fX1lR49esjWrVulT58+0qdPn2qXFan8clMRkYsXL8o999wjfn5+EhISIv/3f/8nBw4cqPRy05q+z0pKSuTtt9+WVq1aiVarlYiICBk8eLDs3r3bMs+RI0ekd+/e4uvra3W5YWXjWIiUXl7aqlUr8fb2lqioKHn00Udtxhapah/HjBkjcXFx1TeQiBQXF8u0adMkISFBvL29JTY2VqZMmWJ1yWV1bVGZqmqKi4ur9LLjir8bavK+SktLE71eL0OHDrVZ31133SX+/v5y/PjxKmus7HJTkdIxJR588EGJjo4Wb29vady4sdx+++3y7bffWuYxf7927txptWxllzKXlJTIyy+/LNHR0eLr6yv9+/eXw4cPS1hYmDzyyCNWy3/88cfSrFkz0Wg0Vuupqt1q+nNAtlQiyvdOUalUWLlypWWM/+XLl2PUqFE4ePCgTaeagIAAREdH49VXX7U5rF9QUAA/Pz+sXbsWt9xyiyt3gQjDhg2r1WWfROQ4mZmZCAkJwRtvvIGXXnpJ6XI8Ur08FdKxY0cYjUakp6dbXVtcXo8ePVBSUoJjx45ZTgn8/fffAGw7NRI5WkFBgdWVJSkpKfj555+txg4hIueq+HMIXO3TYu7oT66n2BGL3NxcSy/6jh074r333kO/fv0QGhqKpk2b4oEHHsDvv/+Od999Fx07dsTFixexYcMGtG/fHkOGDIHJZEKXLl0QEBCA999/HyaTCRMmTEBQUBDWrl2rxC6RB2nUqJHlviKnTp3CggULYDAY8Ndff6FFixZKl0fkERYvXozFixfjtttuQ0BAAH777TcsXboUt956K9asWaN0eZ5LqXMw5vNlFR/mc3xFRUXyyiuvSHx8vHh7e0ujRo3krrvukn379lnWcebMGbn77rslICBAoqKiZOzYsXL58mWF9og8ydixYyUuLk50Op0EBQXJwIEDrc73E5Hz7d69WwYMGCBhYWHi7e0tTZo0kYkTJ0pOTo7SpXm0etHHgoiIiNxDvR3HgoiIiBoeBgsiIiJyGJdfFWIymXD27FkEBgZymFUiIqIGQkSQk5ODmJgYqzsDV+TyYHH27FmbO/ARERFRw5CWlmYZjbUyLg8W5ptipaWlWW57TQ1AXh5QNrw6zp4Fyt2rg4iI3F92djZiY2OtbopZGZcHC/Ppj6CgIAaLhqT8CKhBQQwWREQe6lrdGNh5k4iIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHMblNyEjouqJCEpMAqNJYBKBSQCTCMQEGMU8TSACyzzlvzZJ6TqMIjCZcPV1y3Kl85jnR+k/iHk7ZTUIUPZa6fJls1rWB/N0mF8r/1ysp5WfbvW67bIom8dkknKvly6DCvNb2symDcu/JlW+VlnbV72eqrdRcTu2r1W9YF1ql2pfq7r4apqgViq2W93X59DVOWF/Hbw+h1cIPH3LdQj08Xb4emuCwYI8lskkKCwxIs9gRH5RieV/Q4kJRSUmGEqMMJSYrj6KjSgymmAoNtnMY/66xFgaCkpMpnJfC0qM5uemsuelXxtNgmJj2eumq4GCiKguHu2byGBBZA8RQY6hBFn5xcgqsH5klpuWXVCM7MJi5BcZyx5XA0R+kVHp3ag1lQrQqFRQq1RQqQC1SgWN2vprtar09sal85V+rVbbLqdSASqU/a9SQVW2fqvpZRtVl31tNR9KJ5qfV1wnKsyvLvc1yq2//DJqVfl1lt/W1ecwb7tcm1i1UYX2sn7NekL5123vCF2zbdisp7pt2CxX9W2o7aq9unoqrKj6G1/XgoNXWHE/67w+h9fn4PU5eIV+WuU+3hksqF4REWTkFeFMZgHOXCnAxVwDLuYYcKns/4u5RbiUY8DFXAOKSkwO266/VgM/nRf8tBr4eGmg81ZDq1FD562GzktT7ms1tF6l0yr72lujgkZt/l8FL7UaXmoVvDRlX2tUZc8rTK9kHrW6NBRYBQbV1QBARFQfMViQyxUbTUjLyMexi3k4fjEXaVfycfpKaZA4faUABcU1P5Kg81JD7+sNva83gv1K/w8qe25+BPp4I0CngZ/WC/7m/7Ve8NVq4K8rDRJqNT+oiYgcgcGCnEZEkJZRgANns3DwbBZSLuTi2MVcnLqcj5Jr9COICtIhJtgXUYE+iAjUITxAV/a/1vI8PEAHX63GRXtDREQ1wWBBDnM2swA7T2Zg/+mssjCRjZzCkkrn9fXWIDHSH83CAxAX5ofGwb5oEuKHJiG+aBTsA50XAwMRUUPEYEG1IiI4cSkPW49fxq6TV7DjRAbOZBbYzKfVqNGqUSDaxAShZVQgEiMDkBgRgOggH55+ICJyQwwWZLfXfjyI9al5SM3It5quVgFtYvTo2DQYbRvr0TZGjxZRAfDWcBw2IiJPwWBB1co1lGDD4QtYu/M45pVNW7ojDQVaH3hrVLixaQi6JYSiS0IoOjYNQYCObykiIk/GTwGyUWI0YXPyRXy7+zQ2JafDUGKCb1Gh5fX7uzbBze3jcXNiGPwZJIiIqBx+KpDF2cwCLNuZhm92puF89tUgkRDuj2HXxQCzS5+/OrQt4O+vUJVERFSfMVgQ/r6Qgw83H8P3e89ahpMO9dfinhsbY1jHxmjdKAiq/PxrrIWIiIjBwqP9fSEH76xJxtpDFyzTuiWE4oGb4nBrmyhe8klERHZjsPBA6dmFeG/d3/hmVxpMUjpG/aA20XikTyI6xAYrXR4RETVgDBYexGQSfLX9FGb9koxcQ+nAVYPaROOZgS3RPDJA4eqIiMgdMFh4iKPpOXju2334MzUTANAhNhgvD7keneNDlS2MiIjcCoOFmxMRfLMrDa/+cBCFxSYE6Lzw3KCWGNUtDhqOfElERA7GYOHGcg0lmPLdfvy49ywAoFeLcLw1vD0a6X0VroyIiNwVg4WbOpNZgH8u3okj53OgUavwzK0t8X+9m/H+HERE5FQMFm5ob1om/vn5LlzKNSA8QIeFo29Epzj2pSAiIudjsHAz245fxkOLdyK/yIhW0YH4dGwXNA7mqQ8iInINBgs38r+Uixj/xS4UFpvQs3k4PhzdiTcFIyIil+Knjpv449gl/PPzXSgqMaFfywgseKATfLw5ciYREbkWg4UbOHg2C//6YjeKSky4pXUU5v3jRmi91EqXRUREHoifPg1cWkY+xi7aiVxDCbomhGLOyI4MFUREpBh+AjVg+UUlePjzXbiYY0DLqEB8/GBnnv4gIiJFMVg0UCKC5/+zH8kXchARqMPih7pA7+utdFlEROThGCwaqM9+P4kf956Fl1qF+aNu5GiaRERULzBYNECHz2Vj1uojAID/N+R6dOGNxIiIqJ5gsGhgDCVGPLV8D4qMJiRdH4kxN8crXRIREZGF3cHizJkzeOCBBxAWFgZfX1+0a9cOu3btckZtVInZ61Jw5HwOwvy1mHF3e6hUvPcHERHVH3aNY3HlyhX06NED/fr1w+rVqxEREYGUlBSEhIQ4qz4qZ09aJhZuOQYAePPudogI1ClcERERkTW7gsWsWbMQGxuLRYsWWaYlJCQ4vCiyZTQJ/t+q/RABht0Qg4FtopUuiYiIyIZdp0J++OEHdO7cGSNGjEBkZCQ6duyIjz/+uNplDAYDsrOzrR5kv6+2n8KBM9kI8vHCS0NaK10OERFRpewKFsePH8eCBQvQokULrFmzBo8++iiefPJJfP7551UuM2PGDOj1essjNja2zkV7mos5Bry9JhkA8OygVjwFQkRE9ZZKRKSmM2u1WnTu3Bl//PGHZdqTTz6JnTt3YuvWrZUuYzAYYDAYLM+zs7MRGxuLrKwsBAUF1aF0z/H/Vu3Hl9tS0a6xHqsm9IBGrUCHzbw8ICCg9OvcXMDf3/U1EBGRYrKzs6HX66/5+W3XEYtGjRqhdWvrw/DXX389UlNTq1xGp9MhKCjI6kE1d+xiLpbuSAMAvDTkemVCBRERUQ3ZFSx69OiB5ORkq2l///034uLiHFoUXfX2L8kwmgQDWkXipmZhSpdDRERULbuCxVNPPYVt27bhzTffxNGjR/H111/jo48+woQJE5xVn0f7M/UKfjl4HmoV8PzgVkqXQ0REdE12BYsuXbpg5cqVWLp0Kdq2bYvXX38d77//PkaNGuWs+jzavzekAADuubEJrosKVLgaIiKia7NrHAsAuP3223H77bc7oxYqZ9/pTGxOvgi1CpjQr7nS5RAREdUI7xVST83ZeBQAcOcNjREfziswiIioYWCwqIcOn8vGukMXoOLRCiIiamAYLOqhT/53AgBwW7tGaB4ZoHA1RERENcdgUc9czDHgx71nAQAP9+R9WIiIqGFhsKhnvt6eiiKjCTfEBqNjU941loiIGhYGi3rEUGLEkm2nAADjesQrWwwREVEtMFjUI//ddw6Xcg2ICtLhtnaNlC6HiIjIbgwW9cjnf5wEAIy+KQ7eGn5riIio4eGnVz1x+Fw29p7OgrdGhfu7NlW6HCIiolphsKgnlu8svYNp0vVRCA/QKVwNERFR7TBY1AOGEiNW7TkDALi3S6zC1RAREdUeg0U9sPbgBWTmF6OR3ge9W0QoXQ4REVGtMVjUA9/sKj0NMrxTE2jUKoWrISIiqj0GC4WdvpKP345eAgCM6MTTIERE1LAxWCjsx73nIALc1CwUTcP8lC6HiIioThgsFPZD2X1B7ryhscKVEBER1R2DhYKOpufg8LlseKlVGNw2WulyiIiI6ozBQkE/7D0HAOh9XQSC/bQKV0NERFR3DBYKERHL7dHv6BCjcDVERESOwWChkINns3HiUh50XmoktY5SuhwiIiKHYLBQiPloRdL1UQjQeSlcDRERkWMwWChARPDLwfMAgCHteXt0IiJyHwwWCki+kINTl/Oh81Kjb0sO4U1ERO6DwUIBaw9eAAD0ahEOPy1PgxARkftgsFDA2kOlp0Fubc2xK4iIyL0wWLjYmcwCHDiTDbUKGHB9pNLlEBERORSDhYutK+u02TkuFGEBOoWrISIiciwGCxdbe6i0f8WtbTh2BRERuR8GCxfKyi/G9hMZAIBbOCgWERG5IQYLF/rf0YswmgQtIgMQF+avdDlEREQOx2DhQpuTLwIAx64gIiK3xWDhIiaT4Ne/S4NFn+t4NQgREbknBgsXOXQuGxdzDPDTatAlIUTpcoiIiJyCwcJFzEcrbk4Mg85Lo3A1REREzsFg4SKbk9MBAH1a8jQIERG5LwYLF8gqKMafqZkAgL7XseMmERG5LwYLF/gt5RKMJkFihD9iQ/2ULoeIiMhpGCxc4LejvBqEiIg8A4OFC/x+9DIAoGeLMIUrISIici4GCydLy8hHakY+NGoVusSHKl0OERGRU9kVLKZOnQqVSmX1aNWqlbNqcwtbj5UerejQRI9AH2+FqyEiInIuL3sXaNOmDdavX391BV52r8Kj/H7sEgCgR/NwhSshIiJyPrtTgZeXF6Kjo2s8v8FggMFgsDzPzs62d5MNlojgj7IjFt0T2b+CiIjcn919LFJSUhATE4NmzZph1KhRSE1NrXb+GTNmQK/XWx6xsbG1LrahOZqei4s5Bui81LixKYfxJiIi92dXsOjWrRsWL16MX375BQsWLMCJEyfQq1cv5OTkVLnMlClTkJWVZXmkpaXVueiG4vejpadBusSHwsebw3gTEZH7s+tUyODBgy1ft2/fHt26dUNcXBy++eYb/POf/6x0GZ1OB51OV7cqGyieBiEiIk9Tp8tNg4ODcd111+Ho0aOOqsdtmEyCbcdLg8XNDBZEROQh6hQscnNzcezYMTRq1MhR9biN5As5yC4sgZ9Wg3aN9UqXQ0RE5BJ2BYtnnnkGv/76K06ePIk//vgDd911FzQaDUaOHOms+hqsnSczAAA3Ng2Bl4bjkBERkWewq4/F6dOnMXLkSFy+fBkRERHo2bMntm3bhogI3rGzoh0nSoMFR9skIiJPYlewWLZsmbPqcCsiYjli0SWBl5kSEZHn4DF6Jzh9pQAXsg3w1qjQMZbBgoiIPAeDhROYT4O0bayHr5bjVxARkedgsHAC82mQruxfQUREHobBwgl2nGTHTSIi8kwMFg52KdeA4xfzAACd49m/goiIPAuDhYPtOnkFAHBdVACC/bQKV0NERORaDBYO9ldaabDoFMejFURE5HkYLBxsT2omAPAyUyIi8kgMFg5kNAn2n8kCANzQNFjZYoiIiBTAYOFAf1/IQX6REQE6LyRGBChdDhERkcsxWDjQX2WnQdo30UOjVilbDBERkQIYLBxoT1nHzRtig5UthIiISCEMFg60Jy0TAIMFERF5LgYLB8kpLEZKei4AdtwkIiLPxWDhIPtPZ0EEaBzsi8hAH6XLISIiUgSDhYP8ZT4NwqMVRETkwRgsHOQvy8BYwYrWQUREpCQGCwc5UDYwVvsmwcoWQkREpCAGCwe4lGvA+exCAEDrmCCFqyEiIlIOg4UDHDybDQBoFu6PAJ2XwtUQEREph8HCAQ6eLT0NwqMVRETk6RgsHODgmdIjFm0b6xWuhIiISFkMFg5gPmLRhkcsiIjIwzFY1FF2YTFOXs4HALSJ4RELIiLybAwWdXSorONmjN4Hof5ahashIiJSFoNFHZmvCGnD/hVEREQMFnV1sGxgrLY8DUJERMRgUVeWIxbsuElERMRgUReFxUYcvVh6q3ReakpERMRgUSfJ53NgNAnC/LWICtIpXQ4REZHiGCzqIPl8DgCgVaNAqFQqhashIiJSHoNFHSRfKA0W10UFKlwJERFR/cBgUQd/lwWLlgwWREREABgs6uRI2amQltEMFkRERACDRa1l5BXhYo4BANCCRyyIiIgAMFjUmrnjZpMQXwTovBSuhoiIqH5gsKglc/+KVjwNQkREZMFgUUu8IoSIiMgWg0UtJbPjJhERkY06BYuZM2dCpVJh0qRJDiqnYRAR/M1gQUREZKPWwWLnzp1YuHAh2rdv78h6GoRzWYXIMZTAS61Cs/AApcshIiKqN2oVLHJzczFq1Ch8/PHHCAkJcXRN9Z75NEhCuD+0XjybREREZFarT8UJEyZgyJAhSEpKuua8BoMB2dnZVo+Gztxxk6dBiIiIrNk9AMOyZcvw559/YufOnTWaf8aMGZg2bZrdhdVnf/OKECIiokrZdcQiLS0NEydOxFdffQUfH58aLTNlyhRkZWVZHmlpabUqtD45djEPANAikv0riIiIyrPriMXu3buRnp6OG2+80TLNaDRiy5YtmDt3LgwGAzQajdUyOp0OOp3OMdXWAyKCY+m5AIBEBgsiIiIrdgWLAQMGYP/+/VbTxo0bh1atWuH555+3CRXuKD3HgFxDCdQqIC7MT+lyiIiI6hW7gkVgYCDatm1rNc3f3x9hYWE2092V+WhF01A/6LzcP0gRERHZg9dK2unYxbLTIBE8DUJERFRRnW/LuXnzZgeU0XCYO242Z/8KIiIiGzxiYScesSAiIqoag4Wdrl4R4q9wJURERPUPg4Ud8gwlOJtVCAC8RwgREVElGCzscOJSaf+KMH8tQvy1CldDRERU/zBY2IH9K4iIiKrHYGEH9q8gIiKqHoOFHcyXmvKIBRERUeUYLOxgORXCMSyIiIgqxWBRQ0aT4HhZ583mPGJBRERUKQaLGjp9JR9FJSbovNSICfZVuhwiIqJ6icGihsynQRLC/aFRqxSuhoiIqH5isKih4+y4SUREdE0MFjV08nJpsIgP91O4EiIiovqLwaKGTl7KBwDEh3EMCyIioqowWNSQeTjv+HAGCyIioqowWNRAYbERZ7MKAPCIBRERUXUYLGrg9JV8iAABOi+EB/DmY0RERFVhsKiBE+b+FeF+UKl4qSkREVFVGCxq4KS5fwVPgxAREVWLwaIGTpRdaprAjptERETVYrCoAfMRizgesSAiIqoWg0UNnLpc2scigYNjERERVYvB4hp4qSkREVHNMVhcQ2pG6aWmgT5eCPXnpaZERETVYbC4BvOImwnh/rzUlIiI6BoYLK6Bl5oSERHVHIPFNVjuahrGjptERETXwmBxDZa7mnIMCyIiomtisLgGyxELBgsiIqJrYrCoRkGREeeyCgEACexjQUREdE0MFtU4lVF6tELv640QXmpKRER0TQwW1bh6RQg7bhIREdUEg0U1Tl5mx00iIiJ7MFhUIzWjNFjEhfKIBRERUU0wWFQjrSxYxDJYEBER1QiDRTXMRyyaMlgQERHVCINFFUqMJpy5UnpX06bsvElERFQjDBZVOJdViBKTQKtRIyrQR+lyiIiIGgQGiyqY+1c0CfWFWs27mhIREdWEXcFiwYIFaN++PYKCghAUFITu3btj9erVzqpNUexfQUREZD+7gkWTJk0wc+ZM7N69G7t27UL//v1x55134uDBg86qTzEMFkRERPbzsmfmoUOHWj2fPn06FixYgG3btqFNmzYOLUxpaeaOmwwWRERENWZXsCjPaDRixYoVyMvLQ/fu3aucz2AwwGAwWJ5nZ2fXdpMulcoxLIiIiOxmd+fN/fv3IyAgADqdDo888ghWrlyJ1q1bVzn/jBkzoNfrLY/Y2Ng6FewqlsGxQhgsiIiIasruYNGyZUvs2bMH27dvx6OPPooxY8bg0KFDVc4/ZcoUZGVlWR5paWl1KtgVcgqLkZFXBACIDfVVuBoiIqKGw+5TIVqtFs2bNwcAdOrUCTt37sQHH3yAhQsXVjq/TqeDTqerW5UulpZR2r8i1F+LQB9vhashIiJqOOo8joXJZLLqQ+EO2L+CiIioduw6YjFlyhQMHjwYTZs2RU5ODr7++mts3rwZa9ascVZ9ikjjpaZERES1YlewSE9Px4MPPohz585Br9ejffv2WLNmDW655RZn1aeIq2NYsH8FERGRPewKFp9++qmz6qhXODgWERFR7fBeIZVIYx8LIiKiWmGwqMBkEpzmqJtERES1wmBRwYWcQhQZTfBSq9BIzz4WRERE9mCwqCD1culpkMYhvtDwdulERER2YbCogB03iYiIao/BogJ23CQiIqo9BosKeMSCiIio9hgsKmCwICIiqj0GiwpSM3ipKRERUW0xWJSTX1SCS7mlN1RjHwsiIiL7MViUY75dut7XG3pf3i6diIjIXgwW5fCupkRERHXDYFFOquVSU464SUREVBsMFuWkcgwLIiKiOmGwKIenQoiIiOqGwaIcjmFBRERUNwwWZUSEwYKIiKiOGCzKXMwxwFBigloFxASz8yYREVFtMFiUMR+tiAn2hbeGzUJERFQb/AQtw9MgREREdcdgUSaN9wghIiKqMwaLMhzDgoiIqO4YLMqkMVgQERHVGYNFGfaxICIiqjsGCwCFxUaczy4EwGBBRERUFwwWAE5fKe24GaDzQogfb5dORERUWwwWsO5foVKpFK6GiIio4WKwQPn+FRxxk4iIqC4YLMCOm0RERI7CYAEGCyIiIkdhsADHsCAiInIUjw8WIsJgQURE5CAeHywy8oqQV2SESgU05u3SiYiI6sTjg4W5f0V0kA98vDUKV0NERNSwMVjwNAgREZHDeHywSOMVIURERA7j8cGCl5oSERE5DoMFgwUREZHDeHywSMsovQEZ+1gQERHVnUcHi6ISE85mlQYLHrEgIiKqO7uCxYwZM9ClSxcEBgYiMjISw4YNQ3JysrNqc7qzmQUQAXy9NQgP0CpdDhERUYNnV7D49ddfMWHCBGzbtg3r1q1DcXExbr31VuTl5TmrPqe6eqmpL2+XTkRE5ABe9sz8yy+/WD1fvHgxIiMjsXv3bvTu3duhhbkCO24SERE5ll3BoqKsrCwAQGhoaJXzGAwGGAwGy/Ps7Oy6bNKheI8QIiIix6p1502TyYRJkyahR48eaNu2bZXzzZgxA3q93vKIjY2t7SYdjkcsiIiIHKvWwWLChAk4cOAAli1bVu18U6ZMQVZWluWRlpZW2006HIMFERGRY9XqVMjjjz+On376CVu2bEGTJk2qnVen00Gn09WqOGcSEaReZrAgIiJyJLuChYjgiSeewMqVK7F582YkJCQ4qy6nyyooRo6hBADQJITBgoiIyBHsChYTJkzA119/je+//x6BgYE4f/48AECv18PX19cpBTqL+TRIZKAOvlreLp2IiMgR7OpjsWDBAmRlZaFv375o1KiR5bF8+XJn1ec07F9BRETkeHafCnEXvEcIERGR43nsvUJSOYYFERGRw3lssEjjqRAiIiKH89hgwT4WREREjueRwaLEaMKZTN4unYiIyNE8MlicyyqE0STQeqkRGVj/Bu8iIiJqqDwyWFg6bob4Qq3m7dKJiIgcxaODBU+DEBERORaDBRERETmMRwcLjmFBRETkWB4ZLE4zWBARETmFRwYLngohIiJyDo8LFtmFxbiSXwyARyyIiIgczeOChXko7zB/LQJ0dt2DjYiIiK7BY4MFj1YQERE5nscFC/avICIich4GCyIiInIYDwwWvPkYERGRs3hcsGAfCyIiIufxqGBhNAlOXzEHC1+FqyEiInI/HhUsLmQXotgo8FKr0EjPYEFERORoHhUszB03m4T4QsPbpRMRETmcRwYL9q8gIiJyDo8KFmm81JSIiMipPCpYcAwLIiIi52KwICIiIofxqGDBMSyIiIicy2OCRZ6hBJdyiwAATcMYLIiIiJzBY4JFWtnAWHpfbwT5eCtcDRERkXvynGDBe4QQERE5nccEC3bcJCIicj6PCRbsuElEROR8HhMseMSCiIjI+RgsiIiIyGE8IliYTMLhvImIiFzAI4LFxVwDDCUmaNQqNAr2UbocIiIit+URwcJ8GiQm2AfeGo/YZSIiIkV4xKds6mWeBiEiInIFzwgW5ktNQxgsiIiInMkjgoV5OG+OYUFERORcdgeLLVu2YOjQoYiJiYFKpcKqVaucUJZj8YoQIiIi17A7WOTl5aFDhw6YN2+eM+pxCo5hQURE5Bpe9i4wePBgDB482Bm1OEVhsREXsg0AGCyIiIicze5gYS+DwQCDwWB5np2d7exNWjld1r8iUOeFYD/eLp2IiMiZnN55c8aMGdDr9ZZHbGysszdpJbXczcdUKpVLt01ERORpnB4spkyZgqysLMsjLS3N2Zu0wjEsiIiIXMfpp0J0Oh10Op2zN1Ol1IwCAEDTMAYLIiIiZ3P7cSyuDo7lq3AlRERE7s/uIxa5ubk4evSo5fmJEyewZ88ehIaGomnTpg4tzhHSMjg4FhERkavYHSx27dqFfv36WZ4//fTTAIAxY8Zg8eLFDivMEUTEMuom+1gQERE5n93Bom/fvhARZ9TicJfzipBfZIRKBTTmqRAiIiKnc+s+Fub+FY2CfKDz0ihcDRERkftz62DB/hVERESu5dbBgmNYEBERuZZ7BwvefIyIiMilPCNYcHAsIiIil3DrYME+FkRERK7ltsHCUGLEuexCAEBsCIMFERGRK7htsDhzpQAigK+3BuEBWqXLISIi8ghuGyzSrpTdfIy3SyciInIZtw0WqexfQURE5HJuGyzSeKkpERGRy7ltsLg6OBbvEUJEROQq7hssOIYFERGRy7llsBARngohIiJSgFsGi8z8YuQYSgAATTiGBRERkcu4ZbAwnwaJDNTBx5u3SyciInIVtw4WPA1CRETkWgwWRERE5DBuGSxOX+HgWEREREpwy2DBIxZERETKcO9gwTEsiIiIXMrtgkWx0YSzmaW3S+cRCyIiItdyu2BxLrMQRpNA56VGRIBO6XKIiIg8itsFi/J3NVWrebt0IiIiV3LbYMHTIERERK7ntsEiNoR3NSUiInI1twsWaRkcw4KIiEgpbhcseCqEiIhIOW4XLNKucAwLIiIipbhVsMgqKEZmfjEAIJa3SyciInI5twoW5v4V4QFa+Ou8FK6GiIjI87hlsGDHTSIiImW4VbBgx00iIiJlMVgQERGRw7hlsGDHTSIiImW4VbBgHwsiIiJluU2wMJoEZzILAHAMCyIiIqW4TbA4n12IYqPAW6NCdJCP0uUQERF5JLcJFqmXS0+DNAnxg4a3SyciIlKE2wQL9q8gIiJSXq2Cxbx58xAfHw8fHx9069YNO3bscHRddrt6qSlvl05ERKQUu4PF8uXL8fTTT+PVV1/Fn3/+iQ4dOmDgwIFIT093Rn01xjEsiIiIlGd3sHjvvfcwfvx4jBs3Dq1bt8aHH34IPz8/fPbZZ5XObzAYkJ2dbfVwBgYLIiIi5dkVLIqKirB7924kJSVdXYFajaSkJGzdurXSZWbMmAG9Xm95xMbG1q3iKrCPBRERkfLsChaXLl2C0WhEVFSU1fSoqCicP3++0mWmTJmCrKwsyyMtLa321VZBRPBo30SM6R7HIxZEREQKcvq9xXU6HXQ6nVO3oVKp8HCvZk7dBhEREV2bXUcswsPDodFocOHCBavpFy5cQHR0tEMLIyIioobHrmCh1WrRqVMnbNiwwTLNZDJhw4YN6N69u8OLIyIioobF7lMhTz/9NMaMGYPOnTuja9eueP/995GXl4dx48Y5oz4iIiJqQOwOFvfddx8uXryIV155BefPn8cNN9yAX375xaZDJxEREXkelYiIKzeYnZ0NvV6PrKwsBAUFuXLTVBd5eUBAQOnXubmAv7+y9RARkUvV9PPbbe4VQkRERMpjsCAiIiKHYbAgIiIih2GwICIiIodhsCAiIiKHYbAgIiIih2GwICIiIodhsCAiIiKHcfrdTSsyj8eVnZ3t6k1TXeTlXf06OxswGpWrhYiIXM78uX2tcTVdHixycnIAALGxsa7eNDlKTIzSFRARkUJycnKg1+urfN3lQ3qbTCacPXsWgYGBUKlUDltvdnY2YmNjkZaWxqHCy7BNbLFNbLFNKsd2scU2seVJbSIiyMnJQUxMDNTqqntSuPyIhVqtRpMmTZy2/qCgILf/5tqLbWKLbWKLbVI5tosttoktT2mT6o5UmLHzJhERETkMgwURERE5jNsEC51Oh1dffRU6nU7pUuoNtokttokttknl2C622Ca22Ca2XN55k4iIiNyX2xyxICIiIuUxWBAREZHDMFgQERGRwzBYEBERkcMwWBAREZHDuE2wmDdvHuLj4+Hj44Nu3bphx44dSpfkFDNmzECXLl0QGBiIyMhIDBs2DMnJyVbzFBYWYsKECQgLC0NAQADuueceXLhwwWqe1NRUDBkyBH5+foiMjMSzzz6LkpISV+6K08ycORMqlQqTJk2yTPPENjlz5gweeOABhIWFwdfXF+3atcOuXbssr4sIXnnlFTRq1Ai+vr5ISkpCSkqK1ToyMjIwatQoBAUFITg4GP/85z+Rm5vr6l1xCKPRiJdffhkJCQnw9fVFYmIiXn/9dasbKnlCm2zZsgVDhw5FTEwMVCoVVq1aZfW6o9pg37596NWrF3x8fBAbG4u33nrL2btWa9W1SXFxMZ5//nm0a9cO/v7+iImJwYMPPoizZ89arcPd2qROxA0sW7ZMtFqtfPbZZ3Lw4EEZP368BAcHy4ULF5QuzeEGDhwoixYtkgMHDsiePXvktttuk6ZNm0pubq5lnkceeURiY2Nlw4YNsmvXLrnpppvk5ptvtrxeUlIibdu2laSkJPnrr7/k559/lvDwcJkyZYoSu+RQO3bskPj4eGnfvr1MnDjRMt3T2iQjI0Pi4uJk7Nixsn37djl+/LisWbNGjh49apln5syZotfrZdWqVbJ371654447JCEhQQoKCizzDBo0SDp06CDbtm2T//3vf9K8eXMZOXKkErtUZ9OnT5ewsDD56aef5MSJE7JixQoJCAiQDz74wDKPJ7TJzz//LC+99JJ89913AkBWrlxp9boj2iArK0uioqJk1KhRcuDAAVm6dKn4+vrKwoULXbWbdqmuTTIzMyUpKUmWL18uR44cka1bt0rXrl2lU6dOVutwtzapC7cIFl27dpUJEyZYnhuNRomJiZEZM2YoWJVrpKenCwD59ddfRaT0h8Db21tWrFhhmefw4cMCQLZu3SoipT9EarVazp8/b5lnwYIFEhQUJAaDwbU74EA5OTnSokULWbdunfTp08cSLDyxTZ5//nnp2bNnla+bTCaJjo6Wt99+2zItMzNTdDqdLF26VEREDh06JABk586dlnlWr14tKpVKzpw547zinWTIkCHy0EMPWU27++67ZdSoUSLimW1S8UPUUW0wf/58CQkJsfrZef7556Vly5ZO3qO6qyxsVbRjxw4BIKdOnRIR928TezX4UyFFRUXYvXs3kpKSLNPUajWSkpKwdetWBStzjaysLABAaGgoAGD37t0oLi62ao9WrVqhadOmlvbYunUr2rVrh6ioKMs8AwcORHZ2Ng4ePOjC6h1rwoQJGDJkiNW+A57ZJj/88AM6d+6MESNGIDIyEh07dsTHH39sef3EiRM4f/68VZvo9Xp069bNqk2Cg4PRuXNnyzxJSUlQq9XYvn2763bGQW6++WZs2LABf//9NwBg7969+O233zB48GAAntkmFTmqDbZu3YrevXtDq9Va5hk4cCCSk5Nx5coVF+2N82RlZUGlUiE4OBgA26Qil9/d1NEuXboEo9Fo9YEAAFFRUThy5IhCVbmGyWTCpEmT0KNHD7Rt2xYAcP78eWi1Wssb3iwqKgrnz5+3zFNZe5lfa4iWLVuGP//8Ezt37rR5zRPb5Pjx41iwYAGefvppvPjii9i5cyeefPJJaLVajBkzxrJPle1z+TaJjIy0et3LywuhoaENsk1eeOEFZGdno1WrVtBoNDAajZg+fTpGjRoFAB7ZJhU5qg3Onz+PhIQEm3WYXwsJCXFK/a5QWFiI559/HiNHjrTczdTT26SiBh8sPNmECRNw4MAB/Pbbb0qXoqi0tDRMnDgR69atg4+Pj9Ll1AsmkwmdO3fGm2++CQDo2LEjDhw4gA8//BBjxoxRuDplfPPNN/jqq6/w9ddfo02bNtizZw8mTZqEmJgYj20Tsk9xcTHuvfdeiAgWLFigdDn1VoM/FRIeHg6NRmPTw//ChQuIjo5WqCrne/zxx/HTTz9h06ZNaNKkiWV6dHQ0ioqKkJmZaTV/+faIjo6utL3MrzU0u3fvRnp6Om688UZ4eXnBy8sLv/76K/7973/Dy8sLUVFRHtcmjRo1QuvWra2mXX/99UhNTQVwdZ+q+7mJjo5Genq61eslJSXIyMhokG3y7LPP4oUXXsD999+Pdu3aYfTo0XjqqacwY8YMAJ7ZJhU5qg3c7ecJuBoqTp06hXXr1lmOVgCe2yZVafDBQqvVolOnTtiwYYNlmslkwoYNG9C9e3cFK3MOEcHjjz+OlStXYuPGjTaH1jp16gRvb2+r9khOTkZqaqqlPbp37479+/db/SCYf1Aqfhg1BAMGDMD+/fuxZ88ey6Nz584YNWqU5WtPa5MePXrYXIb8999/Iy4uDgCQkJCA6OhoqzbJzs7G9u3brdokMzMTu3fvtsyzceNGmEwmdOvWzQV74Vj5+flQq61/5Wk0GphMJgCe2SYVOaoNunfvji1btqC4uNgyz7p169CyZcsGecjfHCpSUlKwfv16hIWFWb3uiW1SLaV7jzrCsmXLRKfTyeLFi+XQoUPyr3/9S4KDg616+LuLRx99VPR6vWzevFnOnTtneeTn51vmeeSRR6Rp06ayceNG2bVrl3Tv3l26d+9ued18aeWtt94qe/bskV9++UUiIiIa7KWVlSl/VYiI57XJjh07xMvLS6ZPny4pKSny1VdfiZ+fn3z55ZeWeWbOnCnBwcHy/fffy759++TOO++s9LLCjh07yvbt2+W3336TFi1aNKhLK8sbM2aMNG7c2HK56XfffSfh4eHy3HPPWebxhDbJycmRv/76S/766y8BIO+995789ddfliscHNEGmZmZEhUVJaNHj5YDBw7IsmXLxM/Pr95eWlldmxQVFckdd9whTZo0kT179lj93i1/hYe7tUlduEWwEBGZM2eONG3aVLRarXTt2lW2bdumdElOAaDSx6JFiyzzFBQUyGOPPSYhISHi5+cnd911l5w7d85qPSdPnpTBgweLr6+vhIeHy+TJk6W4uNjFe+M8FYOFJ7bJjz/+KG3bthWdTietWrWSjz76yOp1k8kkL7/8skRFRYlOp5MBAwZIcnKy1TyXL1+WkSNHSkBAgAQFBcm4ceMkJyfHlbvhMNnZ2TJx4kRp2rSp+Pj4SLNmzeSll16y+nDwhDbZtGlTpb9DxowZIyKOa4O9e/dKz549RafTSePGjWXmzJmu2kW7VdcmJ06cqPL37qZNmyzrcLc2qQuVSLlh54iIiIjqoMH3sSAiIqL6g8GCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHIbBgoiIiByGwYKIiIgchsGCiIiIHOb/A8ivILzRL/4NAAAAAElFTkSuQmCC",
155
+ "text/plain": [
156
+ "<Figure size 640x480 with 1 Axes>"
157
+ ]
158
+ },
159
+ "metadata": {},
160
+ "output_type": "display_data"
161
+ }
162
+ ],
163
+ "source": [
164
+ "plt.plot([sum(min(l, m) for l in lens) for m in range(max(lens) + 1)])\n",
165
+ "plt.title(\"Token coverage as a function of max length\")\n",
166
+ "plt.axvline(x=256, color=\"red\")"
167
+ ]
168
+ },
169
+ {
170
+ "cell_type": "code",
171
+ "execution_count": null,
172
+ "metadata": {},
173
+ "outputs": [],
174
+ "source": []
175
+ }
176
+ ],
177
+ "metadata": {
178
+ "kernelspec": {
179
+ "display_name": "dl3",
180
+ "language": "python",
181
+ "name": "python3"
182
+ },
183
+ "language_info": {
184
+ "codemirror_mode": {
185
+ "name": "ipython",
186
+ "version": 3
187
+ },
188
+ "file_extension": ".py",
189
+ "mimetype": "text/x-python",
190
+ "name": "python",
191
+ "nbconvert_exporter": "python",
192
+ "pygments_lexer": "ipython3",
193
+ "version": "3.10.8"
194
+ },
195
+ "orig_nbformat": 4,
196
+ "vscode": {
197
+ "interpreter": {
198
+ "hash": "90bfda469df5ac7fed8d7e225d563f60a7a7aa420ccfadb091c914debf775e49"
199
+ }
200
+ }
201
+ },
202
+ "nbformat": 4,
203
+ "nbformat_minor": 2
204
+ }
lora-alpaca-zh/README.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - load_in_8bit: True
9
+ - load_in_4bit: False
10
+ - llm_int8_threshold: 6.0
11
+ - llm_int8_skip_modules: None
12
+ - llm_int8_enable_fp32_cpu_offload: False
13
+ - llm_int8_has_fp16_weight: False
14
+ - bnb_4bit_quant_type: fp4
15
+ - bnb_4bit_use_double_quant: False
16
+ - bnb_4bit_compute_dtype: float32
17
+ ### Framework versions
18
+
19
+
20
+ - PEFT 0.5.0.dev0
lora-alpaca-zh/adapter_config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13a039ff9801d1b9df8764ba34a622aadf2abe5b529f5a27dd4cb75c98fbccd7
3
+ size 449
lora-alpaca-zh/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5e1621f48d9ad8feb1d6d31050275f0aafd080c5c07153301fe2f48411f4406
3
+ size 443
lora-alpaca-zh/checkpoint-1000/README.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - load_in_8bit: True
9
+ - load_in_4bit: False
10
+ - llm_int8_threshold: 6.0
11
+ - llm_int8_skip_modules: None
12
+ - llm_int8_enable_fp32_cpu_offload: False
13
+ - llm_int8_has_fp16_weight: False
14
+ - bnb_4bit_quant_type: fp4
15
+ - bnb_4bit_use_double_quant: False
16
+ - bnb_4bit_compute_dtype: float32
17
+ ### Framework versions
18
+
19
+
20
+ - PEFT 0.5.0.dev0
lora-alpaca-zh/checkpoint-1000/adapter_config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13a039ff9801d1b9df8764ba34a622aadf2abe5b529f5a27dd4cb75c98fbccd7
3
+ size 449
lora-alpaca-zh/checkpoint-1000/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5e1621f48d9ad8feb1d6d31050275f0aafd080c5c07153301fe2f48411f4406
3
+ size 443
lora-alpaca-zh/checkpoint-1000/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b532f7596bad1d50de720a32909661dc201e638dac03feba8baf98332cd74ec6
3
+ size 33661637
lora-alpaca-zh/checkpoint-1000/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1a67040b33d5cbc4d9edece2bb5e621ad7304473ef31b6caf823683dc178208
3
+ size 14575
lora-alpaca-zh/checkpoint-1000/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b0534085c08fed776591fb1f31bb06003e729c584d7ed6df962d22309e50e79
3
+ size 627
lora-alpaca-zh/checkpoint-1000/trainer_state.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61a8d7e9e6c98c723b256c7f85a0370798a786586dc4606e7c2ebf2a66866c22
3
+ size 13389
lora-alpaca-zh/checkpoint-1000/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e1fd39451cf6d7fa4f3d2a89305dce48f7ea6420b5830befb0f22e95a7adfdf
3
+ size 3963
lora-alpaca-zh/checkpoint-600/README.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - load_in_8bit: True
9
+ - load_in_4bit: False
10
+ - llm_int8_threshold: 6.0
11
+ - llm_int8_skip_modules: None
12
+ - llm_int8_enable_fp32_cpu_offload: False
13
+ - llm_int8_has_fp16_weight: False
14
+ - bnb_4bit_quant_type: fp4
15
+ - bnb_4bit_use_double_quant: False
16
+ - bnb_4bit_compute_dtype: float32
17
+ ### Framework versions
18
+
19
+
20
+ - PEFT 0.5.0.dev0
lora-alpaca-zh/checkpoint-600/adapter_config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13a039ff9801d1b9df8764ba34a622aadf2abe5b529f5a27dd4cb75c98fbccd7
3
+ size 449
lora-alpaca-zh/checkpoint-600/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5e1621f48d9ad8feb1d6d31050275f0aafd080c5c07153301fe2f48411f4406
3
+ size 443
lora-alpaca-zh/checkpoint-600/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:211505a9cd519e87d58d17032d8af9f0f20bf29e6de71a163cd83fe39e312fdd
3
+ size 33661637
lora-alpaca-zh/checkpoint-600/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1feeb985f08038725ec233c2cd7d85ba57b2db8645f6eef5b1c78448314f339
3
+ size 14575
lora-alpaca-zh/checkpoint-600/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4900a9fc3293f841025e13baad2479701f96924e68a83ac7596fe416cbfd30b1
3
+ size 627
lora-alpaca-zh/checkpoint-600/trainer_state.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c10fb48f171d16c9822df66d0d0e877a0053d883a0b4c428cbb3e606e6ab368
3
+ size 8180
lora-alpaca-zh/checkpoint-600/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e1fd39451cf6d7fa4f3d2a89305dce48f7ea6420b5830befb0f22e95a7adfdf
3
+ size 3963
lora-alpaca-zh/checkpoint-800/README.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - load_in_8bit: True
9
+ - load_in_4bit: False
10
+ - llm_int8_threshold: 6.0
11
+ - llm_int8_skip_modules: None
12
+ - llm_int8_enable_fp32_cpu_offload: False
13
+ - llm_int8_has_fp16_weight: False
14
+ - bnb_4bit_quant_type: fp4
15
+ - bnb_4bit_use_double_quant: False
16
+ - bnb_4bit_compute_dtype: float32
17
+ ### Framework versions
18
+
19
+
20
+ - PEFT 0.5.0.dev0
lora-alpaca-zh/checkpoint-800/adapter_config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13a039ff9801d1b9df8764ba34a622aadf2abe5b529f5a27dd4cb75c98fbccd7
3
+ size 449
lora-alpaca-zh/checkpoint-800/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e5e1621f48d9ad8feb1d6d31050275f0aafd080c5c07153301fe2f48411f4406
3
+ size 443
lora-alpaca-zh/checkpoint-800/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9843b530770a463007bc5d4ce900e858ef72be4dcf5507dde2a58e88b586b72d
3
+ size 33661637
lora-alpaca-zh/checkpoint-800/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff7856db456f6c08af0a0ef1ff4dedbc5c87c2003cf29273ea1acd1eed891495
3
+ size 14575
lora-alpaca-zh/checkpoint-800/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dca2b4f8be4d4534a63540d7d1b72995e56e8f71a33bac731d65d4f333b57a7f
3
+ size 627
lora-alpaca-zh/checkpoint-800/trainer_state.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9333288f02a81b7e4d0c3dce1ada672466880377a44d7efc104a2a85f3c37996
3
+ size 10795
lora-alpaca-zh/checkpoint-800/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3e1fd39451cf6d7fa4f3d2a89305dce48f7ea6420b5830befb0f22e95a7adfdf
3
+ size 3963
pyproject.toml ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ [tool.black]
2
+ line-length = 79
3
+
4
+ [tool.isort]
5
+ include_trailing_comma = true
6
+ line_length = 79
7
+ multi_line_output = 3
8
+ profile = "black"
requirements.txt ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ accelerate
2
+ appdirs
3
+ loralib
4
+ bitsandbytes
5
+ black
6
+ black[jupyter]
7
+ datasets
8
+ fire
9
+ git+https://github.com/huggingface/peft.git
10
+ transformers>=4.28.0
11
+ sentencepiece
12
+ gradio
templates/README.md ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Prompt templates
2
+
3
+ This directory contains template styles for the prompts used to finetune LoRA models.
4
+
5
+ ## Format
6
+
7
+ A template is described via a JSON file with the following keys:
8
+
9
+ - `prompt_input`: The template to use when input is not None. Uses `{instruction}` and `{input}` placeholders.
10
+ - `prompt_no_input`: The template to use when input is None. Uses `{instruction}` placeholders.
11
+ - `description`: A short description of the template, with possible use cases.
12
+ - `response_split`: The text to use as separator when cutting real response from the model output.
13
+
14
+ No `{response}` placeholder was used, since the response is always the last element of the template and is just to be concatenated to the rest.
15
+
16
+ ## Example template
17
+
18
+ The default template, used unless otherwise specified, is `alpaca.json`
19
+
20
+ ```json
21
+ {
22
+ "description": "Template used by Alpaca-LoRA.",
23
+ "prompt_input": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Input:\n{input}\n\n### Response:\n",
24
+ "prompt_no_input": "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:\n",
25
+ "response_split": "### Response:"
26
+ }
27
+
28
+ ```
29
+
30
+ ## Current templates
31
+
32
+ ### alpaca
33
+
34
+ Default template used for generic LoRA fine tunes so far.
35
+
36
+ ### alpaca_legacy
37
+
38
+ Legacy template used by the original alpaca repo, with no `\n` after the response field. Kept for reference and experiments.
39
+
40
+ ### alpaca_short
41
+
42
+ A trimmed down alpaca template which seems to perform just as well and spare some tokens. Models created with the default template seem to be queryable by the short tempalte as well. More experiments are welcome.
43
+
44
+ ### vigogne
45
+
46
+ The default alpaca template, translated to french. This template was used to train the "Vigogne" LoRA and is to be used to query it, or for extra fine tuning.
templates/alpaca.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ee8b4f807e99e4d5ed4248abebdee4bbe29395fb7485a3650c3a8905d8d4aae
3
+ size 542
templates/alpaca_legacy.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c3ae6b638102138f26fa6d06937e45718a07569b6febac325700ec2dd3d5916
3
+ size 561
templates/alpaca_short.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fc606be3541cfef8f55fc2e9133986e5b8f679607435cdeacf1d9a2c662e85a
3
+ size 281
templates/vigogne.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e53ba20125d9c6e818b4380719ee721835cc8fdae0c107a677fd978ed3a016e
3
+ size 605
utils/README.md ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Directory for helpers modules
2
+
3
+ ## prompter.py
4
+
5
+ Prompter class, a template manager.
6
+
7
+ `from utils.prompter import Prompter`
8
+
9
+ ## callbacks.py
10
+
11
+ Helpers to support streaming generate output.
12
+
13
+ `from utils.callbacks import Iteratorize, Stream`
utils/__init__.py ADDED
File without changes
utils/__pycache__/__init__.cpython-311.pyc ADDED
Binary file (144 Bytes). View file
 
utils/__pycache__/__init__.cpython-39.pyc ADDED
Binary file (126 Bytes). View file
 
utils/__pycache__/callbacks.cpython-311.pyc ADDED
Binary file (3.94 kB). View file
 
utils/__pycache__/callbacks.cpython-39.pyc ADDED
Binary file (2.65 kB). View file
 
utils/__pycache__/prompter.cpython-311.pyc ADDED
Binary file (2.8 kB). View file
 
utils/__pycache__/prompter.cpython-39.pyc ADDED
Binary file (1.62 kB). View file
 
utils/callbacks.py ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24bc1c6507ba1f112afea9194cb5e876a7ceed539e4a5d05c9b888b5634dd8f6
3
+ size 1833
utils/prompter.py ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c2ff7a15a8e33ba8b70938a53b23d3bdba10364387916d599d874768b806302
3
+ size 1669