|
--- |
|
language: |
|
- en |
|
license: wtfpl |
|
library_name: transformers |
|
tags: |
|
- code |
|
- text-generation-inference |
|
datasets: |
|
- flytech/python-codes-25k |
|
- espejelomar/code_search_net_python_10000_examples |
|
metrics: |
|
- accuracy |
|
pipeline_tag: text-generation |
|
--- |
|
# Model Card for GPT_2_CODE |
|
-Goal is to create a small GPT2 python coder |
|
# Table of Contents |
|
- [Model Card for GPT_2_CODE](#model-card-for--model_id-) |
|
- [Table of Contents](#table-of-contents) |
|
- [Table of Contents](#table-of-contents-1) |
|
- [Model Details](#model-details) |
|
- [Model Description](#model-description) |
|
- [Uses](#uses) |
|
- [Direct Use](#direct-use) |
|
- [Downstream Use [Optional]](#downstream-use-optional) |
|
- [Out-of-Scope Use](#out-of-scope-use) |
|
- [Bias, Risks, and Limitations](#bias-risks-and-limitations) |
|
- [Recommendations](#recommendations) |
|
- [Training Details](#training-details) |
|
- [Training Data](#training-data) |
|
- [Training Procedure](#training-procedure) |
|
- [Preprocessing](#preprocessing) |
|
- [Speeds, Sizes, Times](#speeds-sizes-times) |
|
- [Evaluation](#evaluation) |
|
- [Testing Data, Factors & Metrics](#testing-data-factors--metrics) |
|
- [Testing Data](#testing-data) |
|
- [Factors](#factors) |
|
- [Metrics](#metrics) |
|
- [Results](#results) |
|
- [Model Examination](#model-examination) |
|
- [Environmental Impact](#environmental-impact) |
|
- [Technical Specifications [optional]](#technical-specifications-optional) |
|
- [Model Architecture and Objective](#model-architecture-and-objective) |
|
- [Compute Infrastructure](#compute-infrastructure) |
|
- [Hardware](#hardware) |
|
- [Software](#software) |
|
- [Citation](#citation) |
|
- [Glossary [optional]](#glossary-optional) |
|
- [More Information [optional]](#more-information-optional) |
|
- [Model Card Authors [optional]](#model-card-authors-optional) |
|
- [Model Card Contact](#model-card-contact) |
|
- [How to Get Started with the Model](#how-to-get-started-with-the-model) |
|
# Model Details |
|
## Model Description |
|
WIP,Goal is to create a small GPT2 python coder |
|
- **Developed by:** C, o, d, e, M, o, n, k, e, y |
|
- **Shared by [Optional]:** More information needed |
|
- **Model type:** Language model |
|
- **Language(s) (NLP):** eng |
|
- **License:** wtfpl |
|
- **Parent Model:** More information needed |
|
- **Resources for more information:** More information needed |
|
- [GitHub Repo](None) |
|
- [Associated Paper](None) |
|
# Uses |
|
coding assistant |
|
## Direct Use |
|
generate python code snippets |
|
## Downstream Use [Optional] |
|
semi auto coder |
|
## Out-of-Scope Use |
|
describe code |
|
Keep Finetuning on question/python datasets |
|
# Training Details |
|
|
|
## Training Data |
|
flytech/python-codes-25k |
|
espejelomar/code_search_net_python_10000_examples |
|
## Training Procedure |
|
Train/Val/Scheduler |
|
### Preprocessing |
|
More information needed |
|
### Speeds, Sizes, Times |
|
Epochs 3 |
|
# "flytech/python-codes-25k" |
|
Training Loss: 0.4007 |
|
Validation Loss: 0.5526 |
|
Epochs 3 |
|
|
|
# "espejelomar/code_search_net_python_10000_examples" |
|
--Starting Loss: 2.0862 |
|
-Epoch 1/4 | Training Loss: 1.5355 | Validation Loss: 1.1723 |
|
-Epoch 2/4 | Training Loss: 1.0501 | Validation Loss: 1.0702 |
|
-Epoch 3/4 | Training Loss: 0.9804 | Validation Loss: 1.0798 |
|
-Epoch 4/4 | Training Loss: 0.9073 | Validation Loss: 1.0772 |
|
|
|
# Evaluation |
|
Manual comparison with base model |
|
### Testing Data |
|
flytech/python-codes-25k |
|
espejelomar/code_search_net_python_10000_examples |
|
### Factors |
|
80/20 train/val |
|
### Metrics |
|
train/validate |
|
lr scheduling |
|
## Results |
|
Better in python code generation as base gpt2-medium model |
|
# Model Examination |
|
More information needed |
|
# Environmental Impact |
|
- **Hardware Type:** CPU and Colab T4 |
|
- **Hours used:** 4 |
|
- **Cloud Provider:** Google Colab |
|
- **Compute Region:** NL |
|
## Model Architecture and Objective |
|
gpt2 |
|
## Compute Infrastructure |
|
More information needed |
|
### Hardware |
|
CPU and Colab T4 |
|
### Software |
|
pytorch, custom python |
|
# More Information [optional] |
|
Experimental |
|
# Model Card Authors [optional] |
|
CodeMonkeyXL |
|
# Model Card Contact |
|
K00B404 huggingface |
|
# How to Get Started with the Model |
|
Use the code below to get started with the model. |
