Upload LlamaForCausalLM

README.md

@@ -0,0 +1,201 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "/workspace/Llama2-70B/FT",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.38.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,10 @@
+{
+  "bos_token_id": 1,
+  "do_sample": true,
+  "eos_token_id": 2,
+  "max_length": 4096,
+  "pad_token_id": 0,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.38.0.dev0"
+}

model-00001-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd53bffad672c1aecb14dd65d12f007e12ebc2228f99fc030f02bf3e0404b791
+size 4806739440

model-00002-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36b0ed2a3afb55ed7461c313704a0b191747fd20e7ff09acbb0cb9d5423e8940
+size 4630578440

model-00003-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9dc02a30b95c017a6cc478350afe9543fc16c2248f84d6f2880724f48a15936a
+size 4362142864

model-00004-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c283d8f6511d90796546cd2251cca239da3e3a2372b99d7ff00da2d00594302
+size 4966188864

model-00005-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1071231035a3995ef6139c05a9a47be2d68dc04aca89d6368f594bf377adba6
+size 4362142864

model-00006-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49abc698f3034d2fca827c1c9143a13258e408148fa49fe18ff64e07255292e6
+size 4362142864

model-00007-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:690ae5401bb4b9f2e1fa2b19b42f6265536e773de0539d594a6a781a1f4f28aa
+size 4966188864

model-00008-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58a97a878fa2767f7dee26baeaf5e48703696936b1a9a7e9cf3254423382313e
+size 4362142880

model-00009-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59360c6feaac0c0a03f6d6685333a3d3a69741f3f98eba49181cd814dd5a9f43
+size 4362142872

model-00010-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a133fbfa5279d861a98fc2dd87388770ecfc176e81f4e441c8fac714938ebec
+size 4966188880

model-00011-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79dacf1cda9f65cb515eb26c4126223393e7c49f341252c2b57db13ccfda7335
+size 4362142872

model-00012-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39b2cdfe6f3b1f919bd6b1ce77b277a14ff1934e89e18945c1b123b7de93a255
+size 4362142872

model-00013-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d280b5f4f9d1b730716624a4875c41651e288958b012cb49c8e68185c5d4ce51
+size 4966188880

model-00014-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d82adaa8880d1196324a2fea9985809d1f1d9dfa900c0e0239ec9349ef76f3d7
+size 4362142872

model-00015-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a58a98b625befafcb67e92f4e564b009fb53a046d0dcb29bf8e009a0d2cdd77
+size 4362142872

model-00016-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:710f7e8db78a2a39574bbab16ddf420f7a282f7fd98171c7ee5cb4e47eadeaba
+size 4966188880

model-00017-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6923c9478d2f98c49be7c9d1be3d2d7aefdb655e47cae04572d9ca245a59c44
+size 4362142872

model-00018-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74640537d0369b08ca8a469d5dc29e9eeba32c4c4915cfcd622cde79bc86d2ef
+size 4362142872

model-00019-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1ba6eea04282c5892e1e1189f7070b300993662f54a1da2da5476ef6a8d3571
+size 4966188880

model-00020-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:154040c12792a18f04ab72b5bf8fff865646fe44df418e778fab12665c94b189
+size 4362142872

model-00021-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:54d3aaa7ab2d26de3e04a04d35baea3b3f25efa0897f669cde512ba6657128c8
+size 4362142872

model-00022-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bbd71741ceab9eb9608cceb962933e21efccc1e53924b2c1bc571952b0c1d987
+size 4966188880

model-00023-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e00c3d61501e309bcd2857b5803e620fe29504ee61cf50d6e77095c8535642e
+size 4362142872

model-00024-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f7f75f906c8c3246ded63c35bf31a074a6772fc78127d0c8e49f8411bfa0161
+size 4362142872

model-00025-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5defb2c0e777cd9eeaafb015720a3e5d1899e48303630b2ec0c19a4a7c9b57c
+size 4966188880

model-00026-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:715e786f0884d984f94ed5580b010c8738522d0a45da0995e0b5858e949ee95d
+size 4362142872

model-00027-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84965ad6ebcb9fc6eea0ab1f404093a83cb4e82821e5232534721b50e1f850dc
+size 4362142872

model-00028-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d3d9f74c123a1aa1787f4bfb3dc59e2e86b7bde787a2f5ced8240e05f80f840
+size 4966188880

model-00029-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76291c9ec8b1dacc7542f7d8cc31e1fd59b6434e2dfac414fabedcb44247edef
+size 4362142872

model-00030-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1442eeba11be283bc49d515ea56974f23158f604bd3ad219bc72a7956bf970dc
+size 4362142872

model-00031-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b5304c0c26144f4c1d6e27bc7b141aba877832d00cb0bbc8f7233ff6ddc29d1
+size 4966188880

model-00032-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3de1a7489f79fde0cc257ee05fb02d0e2f6fb652f849c3f4048f76c4be3be7bc
+size 4362142872

model-00033-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66ecc053ec32e8b7ca313ec57e0037bad94aed8e028b6db24f7ea7c53bfbef7e
+size 4362142872

model-00034-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a621f36108588990c5502eba2621978afd88fbec270e339c426d6d3b7d354add
+size 4966188880

model-00035-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51c2c6123f0ecdbcaec9bcedd045129f7dc51a162a0d581895ce50bbaf7186d1
+size 4362142872

model-00036-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a47824541f0482149205c1e5a11febdb12772f676f78a293b5ff2d2e55b1d5a0
+size 4362142872

model-00037-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cc9f9de9abb20875cba0f25e3180854e5177e1ab23a03759445e2fe36e608e2
+size 4966188880

model-00038-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a6fd2cc8cc400ee5a027018a2248ea8dab4c5128bc4cc5faa9d547ed95bff37
+size 4362142872

model-00039-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5924a38338075132298090288fba3b3cce9f494b73837b78d78ce9ddebb7c698
+size 4362142872

model-00040-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:143cb228263733142287e5e1ea7dcc0786730391f21110dd4b4eb3ffb7b67e98
+size 4966188880

model-00041-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:62cc5088bfb59aec834bec23dd449e6326f3f605a85dc2214ec10b47a8660889
+size 4362142872

model-00042-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95ee09baf15ad92d93e3f91ee9235692397a48aa8c266eec1b5025138bc1eff6
+size 4362142872

model-00043-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59b559df52bce9eb952d69740362114f9e8c210fd22ac38ab0659acfc80e4104
+size 4966188880

model-00044-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6f5ebb4c57ee2a08540844024234e24df70f1b81743ce56488f4ef566b0f8b3
+size 4362142872

model-00045-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa0cdb72f3bcb91d4c6a4f2f6c8ec76600d22b9e3c47f3d757d9a0013c7caea1
+size 4362142872

model-00046-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bb8acbf2db88813241e8321b99cc1a5bb51cd1ec83afba5e314ddbd7d578745
+size 4966188880

model-00047-of-00061.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44355fbb7bfd019f6f2bb2b8539ebb3ac538ca5db6639192997e9e2d6db35495
+size 4362142872