Upload Qwen2ForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "/workspace/.cache/huggingface/hub/models--Qwen--Qwen2-72B-Instruct/snapshots/1af63c698f59c4235668ec9c1395468cb7cd7e79",
+  "_name_or_path": "ellamind/qwen2-72b-instruct-awq-de",
   "architectures": [
     "Qwen2ForCausalLM"
   ],
@@ -17,8 +17,13 @@
   "num_hidden_layers": 80,
   "num_key_value_heads": 8,
   "quantization_config": {
+    "backend": "autoawq",
     "bits": 4,
+    "do_fuse": false,
+    "exllama_config": null,
+    "fuse_max_seq_len": null,
     "group_size": 128,
+    "modules_to_fuse": null,
     "modules_to_not_convert": null,
     "quant_method": "awq",
     "version": "gemm",
@@ -28,7 +33,7 @@
   "rope_theta": 1000000.0,
   "sliding_window": 131072,
   "tie_word_embeddings": false,
-  "torch_dtype": "bfloat16",
+  "torch_dtype": "float16",
   "transformers_version": "4.42.3",
   "use_cache": false,
   "use_sliding_window": false,

model-00001-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1ad3d5a279ca70d0476fda184457c32c2db3807e8b6376b48306200232fb8648
-size 4975862208
+oid sha256:da844731a49a2e22de43eb30cd7a6acd7e9ed5308bcb766e2cd6c737dada8d46
+size 4975862192

model-00002-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e8db19cccc2beee96fde6da2bb929f1c60bb33cf0ca2e5e4f462cdcc6ccf41f9
-size 4890455456
+oid sha256:9768e173cfdef4a6c02d3beb6d2b4c8dd5dcf4487e0d656889cb18fdc7c67ffd
+size 4890455440

model-00003-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dae051a0faa5b68fe0fae2197d2e06c663b3d98a994de0baf3c634e606ea9813
-size 4981431832
+oid sha256:f9c2aa56e1cf64503a7d3b6ce07cdaf25559f0e8ac00b67d5c672c315610c536
+size 4981431808

model-00004-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9124059b54da49d48cd393cb8ebd14f90b77da86c140782f2e7a7a519fbf1d0f
-size 4972694040
+oid sha256:1288b33fd35de53df3cb199180b5e067bf4067f9411d643de52ebabf70650215
+size 4972694016

model-00005-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:361d772e6938ff508cbc840a2f45f32f37fa016dc50ef6c81ed12018eb2d9385
-size 4890422568
+oid sha256:4376868ec29c7250ddff11798e270d5a0ec8fd174d0e1791a456fcd668768cbf
+size 4890422552

model-00006-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0e84abfa378e3f432bd667495bf988065c66d54668f45165e285097bc7fe1d40
-size 4890455584
+oid sha256:e27eebb39f5b2eebe964d8d4f3a5ee6864fa391888df4d5aa7e5ebd37938f86a
+size 4890455560

model-00007-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:806fcc221c54b91a8811499b7cff8e632375c10dc2c5bd757db0640edd8714a2
-size 4890455584
+oid sha256:ee8800c8ba73fc80f2ea78b7e6295059aeec7e842c0a75f4fea06cd1e086a9ee
+size 4890455560

model-00008-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ce6ca289a537b2697800c42560d8c694e74b44c3d8431182ce7344628061ab69
-size 4481817984
+oid sha256:c61493d1ab181e788db3d399722b74dc027974588bb9b7ce32077d5ae32f5693
+size 4481817968

model-00009-of-00009.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3f09a8b10be984a06c4d8118f1c0c0370ff97ccec72a584aead0d14639b1683a
+oid sha256:a2daf3b842c34c1935313c44d1cedda4ac3fe209bb4fb2d6d22c48ed03febf56
 size 2491416704