metadata

license: apache-2.0
language:
  - en
library_name: transformers

Genstruct 7B

Genstruct 7B is an instruction-generation model, inspired by Ada-Instruct:

Previous methods largely rely on in-context approaches to generate instructions, while Ada-Instruct trained a custom instruction-generation model.

Inspired by this, we took this approach further by grounding the generations in user-provided context passages. Further, the model is trained to generate questions involving complex scenarios that require detailed reasoning, allowing for models trained on the generated data to reason step-by-step.

	ChatGPT	Few-shot prompting	RAG	Ada-Instruct	Genstruct
Open models	❌	☑️	☑️	✅	✅
Grounded generation	❌	❌	✅	❌	✅
Complex questions	❌	❌	❌	☑️	✅
Complex responses	✅	☑️	❌	☑️	✅

An example notebook is provided here, which details how to load and sample from the model.

Alternatively, here's a minimal example:

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_NAME = 'NousResearch/Genstruct-7B'

model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map='cuda', load_in_8bit=True)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

msg =[{
    'title': 'p-value',
    'content': "The p-value is used in the context of null hypothesis testing in order to quantify the statistical significance of a result, the result being the observed value of the chosen statistic T {\displaystyle T}.[note 2] The lower the p-value is, the lower the probability of getting that result if the null hypothesis were true. A result is said to be statistically significant if it allows us to reject the null hypothesis. All other things being equal, smaller p-values are taken as stronger evidence against the null hypothesis."
}]
inputs = tokenizer.apply_chat_template(msg, return_tensors='pt').cuda()

print(tokenizer.decode(model.generate(inputs, max_new_tokens=512)[0]).split(tokenizer.eos_token)[0])