joshsparkes/trapi-llm

license: apache-2.0 language:

• en pipeline_tag: text-generation

So this is the Trapi LLM

Model Training

• Initially its trained on polycoder while we cant get GPUs.
• Once we get GPUs we'll get something like CodeLlama

download_model.py

• Running this downloads the pretrained model defined within
• Run it with python python download_model.py

main.py

• This is where we'll recieve the call from Trapi Agent and for the frontend/App.js call
• Run it with python main.py

mock_backend.py

• Just for testing the frontend/App.js
• Run it with python mock_backend.py

App.py

• The frontend testing UI for the LLM
• Run it with PORT=3000 npm start

Hosting

• We're hosting this LLM on AWS EC2

• Set the pem with chmod 400 london-key-trapi.pem

• SSH in with ssh -i london-key-trapi.pem ubuntu@ec2-13-40-191-156.eu-west-2.compute.amazonaws.com from backend dir

• It should use venv automatically

• Start the server with uvicorn main:app --host 0.0.0.0 --port 8001 --reload

• whoami: ubuntu

• $HOME: /home/ubuntu