File size: 2,576 Bytes
ba8e8e1 4767495 ba8e8e1 4767495 c083e6f 4767495 4901c6c 4767495 02bcfc4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
license: bigscience-openrail-m
language:
- en
---
GPT-J-Pyg_PPO-6B [GPT-J Pygmalion + GPT-J PPO_HH]
GPT-J-Pyg_PPO-6B is an experimental model containing a parameter-wise 40/60 blend (weighted average PPO_HH:Pygmalion) of the weights of ppo_hh_gpt-j and Pygmalion-6b.
-Intended Merge Value-
As with fine-tuning, merging weights does not add information but transforms it, therefore it is important to consider trade-offs.
Pyg_PPO combines ppo_hh_gpt-j and Pygmalion-6b; both technical
achievements are blended with the intent to elevate the strengths of
both. Datasets of both are linked below to assist in exploratory speculation on which datasets in what quantity and configuration have
the largest impact on the usefulness of a model without the expense of
fine-tuning. Blend was done in FP32 and output in FP16.
-Intended Use-
Research purposes only, intended for responsible use.
Express a conversation in natural language, and Pyg_PPO will do the thing.
Try starting a two line prompt such as:
```
Bot: "Hello, how are you?"
You: "I am doing just fine, thank you."
```
Or any other
topic, and the model will carry on in this back and forth format.
Can also be used as a base to merge with other creative,
technical, or adventure themed models of the same class
(GPT-J & 6b NeoX) and parameter size (6b) to experiment with
the morphology of model weights based on the value added
by instruct.
Merge tested using KoboldAI with Nucleus Sampling Top-P set to 0.9, Temperature at 0.6, and Repetition Penalty at 1.1; extra samplers
disabled.
-Credits To-
Core Model:
https://huggingface.co/EleutherAI/gpt-j-6B
Author:
https://www.eleuther.ai/
Model1; 50% ppo_hh_gpt-j:
https://huggingface.co/reciprocate/ppo_hh_gpt-j
Author Repo:
https://huggingface.co/reciprocate
Related; CarperAI:
https://huggingface.co/CarperAI
Dataset is a variant of the Helpful Harmless assistant themed
dataset and Proximal Policy Optimization, specific datasets
used are unknown; listed repo datasets include:
https://huggingface.co/datasets/reciprocate/summarize_eval_ilql
https://huggingface.co/datasets/reciprocate/hh_eval_ilql
PPO explained:
https://paperswithcode.com/method/ppo
Potential HH-type datasets utilized:
https://huggingface.co/HuggingFaceH4
https://huggingface.co/datasets/Anthropic/hh-rlhf
Model2; 50% Pygmalion-6b:
https://huggingface.co/PygmalionAI/pygmalion-6b
Author Repo:
https://huggingface.co/PygmalionAI
Weight merge Script credit to Concedo:
https://huggingface.co/concedo
Model's card template credit to Digitous:
https://huggingface.co/digitous/GPT-R |