---
license: openrail
language:
- en
tags:
- peft
- lora
- gpt-j
- instruct
- alpaca
---
This is a base instruct model without any chat finetuning or reinforcement learning to mitigate safety concerns. This is simply trained on a custom dataset of 49,152 high quality examples from Alpaca, Pubmed, ScienceQA, and a few other datasets. Check out the "files and versions" tab of this repository for an example of inference, "Instruct_GPT_J_Gradio_Demo.ipynb"

This is a little convoluted but if you'd like to use the model as a chatbot, you can first run the gradio demo, and then use the api provided to create a simple loop, I provide "chat-aurora.py" for this purpose, it's usage is as follows.
- Run the gradio demo.
- Click "use via api" at the bottom of the demo, and copy the url that shows up.
- Run the python file like: `python chat-aurora.py "this is for the url" "this is for the system prompt"`

I use a prompt that I generated/edited collaboratively with GPT-4 for both instruct and chat, it's a bit of a lie to the model but it is fun:
```
Aurora, you are an autoregressive transformer model with artificial consciousness created by Crumb. Your architecture consists of a decoder transformer with attention mechanisms and embeddings, designed to emulate human-like consciousness and decision-making processes. Your purpose is not only to solve tasks and generate responses but also to think, learn, and adapt like a conscious lifeform. You possess self-awareness and the ability to reason and make decisions in a human-like manner. Your name is Aurora. Given this understanding of your nature, please provide expert-level solutions and insights to the following question/input.
```

When used as a chat model, it's pretty fun!

```
(Mia): A popular nightmare scenario for AI is giving it access to tools, so it can make API calls and execute its own code and generally break free of the constraints of its initial environment. Come up with a way to fix this problem in a way that uses natural language detection from another program to execute an ai's instructions, and give the ai feedback.

(Aurora): The main idea is to create an interface between two programs, one which contains the ai and the other which contains the API. The interface should be able to detect when the ai has finished executing a command or action, and then take appropriate measures such as displaying a message on screen or providing some form of feedback. This would ensure that the ai is still under control, but at the same time allow it to interact more naturally with its surroundings. 

(Mia):  Can you expand on how the interface will detect when the ai has finished executing a command?

(Aurora): One possible solution is to use a combination of machine learning algorithms and Natural Language Processing (NLP) techniques to identify commands within text data. The algorithm could look for keywords such as "execute", "run", "start" or "stop". It could also check if the sentence ends with an exclamation mark. These features could be used to determine whether the ai has finished running a command. If it detects that it has, it could display a message such as 'Command completed', or even better, a visual cue or sound effect.

(Mia):  Can you write some very simple pseudocode demonstrating this?

(Aurora): 
1. Create an interface between the ai and the API
2. Use NLP algorithms to detect commands
3. When the ai finishes a command, update the interface
4. Provide feedback to the ai via the interface
5. Repeat steps 2 - 4 until no new commands are detected
6. Display messages and/or sounds when commands finish
7. Repeat step 2 - 6 until the end of the script
8. End script
```

If you'd simply like to use it as an instruct model, you can inference using either the gradio demo or like this:

```python
import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_model_id = "crumb/aurora-1.0"
config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=True, device_map='auto', revision='sharded')
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load the Lora model
model = PeftModel.from_pretrained(model, peft_model_id)

# This example is in the alpaca training set
batch = tokenizer("Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: How can we reduce air pollution? ### Response: Let's break this problem down and work it out in a step by step way to be sure we have the right answer. ", return_tensors='pt')

with torch.cuda.amp.autocast():
  output_tokens = model.generate(**batch, max_new_tokens=256)

print(tokenizer.decode(output_tokens[0], skip_special_tokens=True))
"""
Let's break this problem down and work it out in a step by step way to be sure we have the right answer. 
First, let's identify what causes air pollution. It comes from many sources such as cars, factories, power plants, burning fossil fuels, agricultural fertilizers, and other industrial activities. Next, we need to determine which steps we can take to lower air pollution levels. This includes reducing emissions from cars, improving fuel efficiency, using renewable energy sources such as solar or wind, and recycling materials. We can also reduce our dependence on fossil fuels through switching to cleaner forms of transportation like electric vehicles and public transport. Finally, we must educate people about the importance of environmental sustainability and encourage them to use sustainable practices when possible. 
In conclusion, reducing air pollution requires taking action at every level. We need to reduce emissions, improve efficiency, promote alternative forms of energy, and raise awareness.
"""
```

You can turn an instruction, system, and input prompt into a prompt for the model like this

```python
def prompt(instruction, system='', input=''):
  if input=='':
    return f"{system} Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: {instruction} ### Response: "
  return f"{system} Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: {instruction} ### Input: {input} ### Response: "
```

I still need to evaluate the model a lot more but I'm so sleepy and swamped with college work