File size: 3,986 Bytes
2c75be5 2b9fa33 57535cf 2b9fa33 57535cf 2c75be5 7f5b650 a68b8cb 7f5b650 a68b8cb 7f5b650 a68b8cb 7f5b650 a68b8cb 7f5b650 57535cf 7f5b650 7d5ecbf 55b0410 57535cf 7d5ecbf 7f5b650 a68b8cb 7f5b650 57535cf |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
---
language:
- en
license: wtfpl
library_name: transformers
tags:
- code
- text-generation-inference
datasets:
- flytech/python-codes-25k
- espejelomar/code_search_net_python_10000_examples
metrics:
- accuracy
pipeline_tag: text-generation
---
# Model Card for GPT_2_CODE
-Goal is to create a small GPT2 python coder
# Table of Contents
- [Model Card for GPT_2_CODE](#model-card-for--model_id-)
- [Table of Contents](#table-of-contents)
- [Table of Contents](#table-of-contents-1)
- [Model Details](#model-details)
- [Model Description](#model-description)
- [Uses](#uses)
- [Direct Use](#direct-use)
- [Downstream Use [Optional]](#downstream-use-optional)
- [Out-of-Scope Use](#out-of-scope-use)
- [Bias, Risks, and Limitations](#bias-risks-and-limitations)
- [Recommendations](#recommendations)
- [Training Details](#training-details)
- [Training Data](#training-data)
- [Training Procedure](#training-procedure)
- [Preprocessing](#preprocessing)
- [Speeds, Sizes, Times](#speeds-sizes-times)
- [Evaluation](#evaluation)
- [Testing Data, Factors & Metrics](#testing-data-factors--metrics)
- [Testing Data](#testing-data)
- [Factors](#factors)
- [Metrics](#metrics)
- [Results](#results)
- [Model Examination](#model-examination)
- [Environmental Impact](#environmental-impact)
- [Technical Specifications [optional]](#technical-specifications-optional)
- [Model Architecture and Objective](#model-architecture-and-objective)
- [Compute Infrastructure](#compute-infrastructure)
- [Hardware](#hardware)
- [Software](#software)
- [Citation](#citation)
- [Glossary [optional]](#glossary-optional)
- [More Information [optional]](#more-information-optional)
- [Model Card Authors [optional]](#model-card-authors-optional)
- [Model Card Contact](#model-card-contact)
- [How to Get Started with the Model](#how-to-get-started-with-the-model)
# Model Details
## Model Description
WIP,Goal is to create a small GPT2 python coder
- **Developed by:** C, o, d, e, M, o, n, k, e, y
- **Shared by [Optional]:** More information needed
- **Model type:** Language model
- **Language(s) (NLP):** eng
- **License:** wtfpl
- **Parent Model:** More information needed
- **Resources for more information:** More information needed
- [GitHub Repo](None)
- [Associated Paper](None)
# Uses
coding assistant
## Direct Use
generate python code snippets
## Downstream Use [Optional]
semi auto coder
## Out-of-Scope Use
describe code
Keep Finetuning on question/python datasets
# Training Details
## Training Data
flytech/python-codes-25k
espejelomar/code_search_net_python_10000_examples
## Training Procedure
Train/Val/Scheduler
### Preprocessing
More information needed
### Speeds, Sizes, Times
Epochs 3
# "flytech/python-codes-25k"
Training Loss: 0.4007
Validation Loss: 0.5526
Epochs 3
# "espejelomar/code_search_net_python_10000_examples"
--Starting Loss: 2.0862
-Epoch 1/4 | Training Loss: 1.5355 | Validation Loss: 1.1723
-Epoch 2/4 | Training Loss: 1.0501 | Validation Loss: 1.0702
-Epoch 3/4 | Training Loss: 0.9804 | Validation Loss: 1.0798
-Epoch 4/4 | Training Loss: 0.9073 | Validation Loss: 1.0772
# Evaluation
Manual comparison with base model
### Testing Data
flytech/python-codes-25k
espejelomar/code_search_net_python_10000_examples
### Factors
80/20 train/val
### Metrics
train/validate
lr scheduling
## Results
Better in python code generation as base gpt2-medium model
# Model Examination
More information needed
# Environmental Impact
- **Hardware Type:** CPU and Colab T4
- **Hours used:** 4
- **Cloud Provider:** Google Colab
- **Compute Region:** NL
## Model Architecture and Objective
gpt2
## Compute Infrastructure
More information needed
### Hardware
CPU and Colab T4
### Software
pytorch, custom python
# More Information [optional]
Experimental
# Model Card Authors [optional]
CodeMonkeyXL
# Model Card Contact
K00B404 huggingface
# How to Get Started with the Model
Use the code below to get started with the model. |