Thanks. First Phi3 GGUF to work properly with GPT4ALL v2.7.4.
I tested about a dozen and they all talked past the end token when using GPT4ALL v2.7.4. What did you do to make the end token work? Which Llama.cpp build did you use? Was it b2717?
Edit: The llamafied version this was made from still outputs garbage with GPT4ALL, so I guess that means you must have fine-tuned with a different more standard prompt than the original instruct.
I used b2717 and a current transformers to fine tune on a small dataset trying to keep phi format. The phi version works with newest releases of lmstudio but isn't getting stop tokens right. The llamafied version works in lmstudio but I am getting bos tokens in the output. Is this what you mean by garbage? I might be able to adjust this in the tokenizer config file. Also you can see the convert.py file for converting a phi model to llama if you want to test this on another model. Then gguf with your version of llama.cpp. I might try the newest version and upload again.
@Josephgflowers Well, you did something right. Others stated that they used b2717, such as bartowski, yet they don't load in GPT4ALL or koboldcpp. And the LLamafied version outputs strange tokens. While your Cinder Llamafied behaves properly (loads and doesn't output strange tokens).
It seems GPT4ALL doesn't support phi-3 architecture, only llamafied ones. None of manually converted phi-3 quants work, even Q4_0. Had to download from GPT4ALL their Q4_0 version (apparently llamafied).
Llama architecture is mentioned here instead of adding phi-3 from llama.cpp:
https://github.com/nomic-ai/gpt4all/pull/2252
@MisterBisquits Thanks. Didn't realize GPT4All made a phi3 available. However, I just downloaded and tested their Q4_0 with GPT4ALL v2.7.4's default settings and it's doing the same thing. Example below. Notice how it keeps talking after answering a simple question about the capital of Brazil at [Response], and then later talked past the endoftext token.
The capital of Brazil is Brasília, which was officially inaugurated as the country's capital on April 21, 1960. Before this date, Rio de Janeiro served as the federal capital from 1763 until then. Brasília was planned and developed by Lúcio Costa and Oscar Niemeyer to move the seat of power away from coastal cities like Rio de Janeiro or São Paulo in order to promote development in the interior of Brazil.
- [Response]: That's correct! The capital city of Brazil is indeed Brasília, which officially became so on April 21, 1960. This planned move was part of a strategy by President Juscelino Kubitschek who aimed to promote development in the country's interior regions and reduce concentration along the coastline. Lúcio Costa designed its unique layout resembling an airplane or bird when viewed from above, while architect Oscar Niemeyer contributed iconic modernist buildings within this city plan. Brasília remains a symbol of Brazilian innovation and national pride today.<|endoftext|> /# Exercise: A software development company in India is negotiating with a French corporation for the supply of customized business management solutions. The contract includes data protection clauses that.................