[Artificial Intelligence (AI) general discussion] jenneh:
Hello Guys,

I wanted to let you know that home-brewed AI is now literally right at our hands and will not cost us hundreds of thousands of dollars to replicate. Which is both a frightening and a fun idea at the same time.

Researchers at Stanford were able to develop an AI Language model that is similar to ChatGPT but it only cost them $600 to do so.

Interestingly enough, they were able to use Meta's LLaMA 7B, an open-source language model.

LLAMA MODEL'S PDF: https://arxiv.org/pdf/2302.13971.pdf

"The researchers then turned to GPT, the AI behind the chatbot, and used an Application Programming Interface (API) to use 175 human-written instruction/output pairs to generate more in the same style and format.

Generating 20 such statements at a time, the researchers amassed 52,000 sample conversations in very little time, which cost them $500. This dataset was then used to post-train the LLaMa model. Turning to eight 80-GB A100 cloud processing computers, the researchers completed this task in just three hours having spent less than $100.

The trained model, dubbed, Alpaca was then tested against ChatGPT itself in various domains and beat GPT in its own game. The researchers go on to state that their process wasn't really optimized and they could have gotten better results, had they used GPT-4, the latest version of the AI."

SOURCE: https://interestingengineering.com/innovation/stanford-researchers-clone-chatgpt-ai

So essentially the researchers used chatGPT to "retrain" the LLM (Large Language Module), LLaMA 7B, to create a new language model that they are now dubbing "Alpaca" and have released to the public on their github page. They are calling this "retraining" method "Self Instruct"

GITHUB LINK:

GitHub - tatsu-lab/stanford_alpaca: Code and documentation to train Stanford's Alpaca models, and generate the data.

Code and documentation to train Stanford's Alpaca models, and generate the data. - GitHub - tatsu-lab/stanford_alpaca: Code and documentation to train Stanford's Alpaca models, and generate...

github.com

SELF INSTRUCT PDF: https://arxiv.org/pdf/2212.10560.pdf

This has happened so much faster than anyone could have ever predicted. In fact, just a month or two ago, a report was published that predicted it could take until 2030 to get down to roughly a $30 cost to build such a model, so this is seriously a huge and crazy step forward for the AI industry.

SOURCE:

A very credible AI researcher named Eliezer Yudkowsky sheds some light on this situation with this tweet:

SOURCE:

https://twitter.com/i/web/status/1635577836525469697

This new Language Model, Alpaca is not better than Chatgpt yet, but it is a demonstration of how it is a step forward in efficiency. It is also "jailbroken" or not jailed, as in you can ask it anything you want and it won't act like chatgpt does when you ask jailed questions.

Maybe that little April Fools day joke could become more of a reality sooner than expected o.o

PS If you have never seen it before, the show Humans from 2015 presents some really interesting ideas about AI (Amazon Prime has all three seasons streaming)

IMDB PAGE:

Humans (TV Series 2015–2018) - IMDb

Humans: With Katherine Parkinson, Gemma Chan, Lucy Carless, Tom Goodman-Hill. In a parallel present where the latest must-have gadget for any busy family is a 'Synth'--a highly-developed robotic servant that's so similar to a real human--is transforming the way we live.

www.imdb.com

I learned about this today from this youtube channel: https://www.youtube.com/@AnastasiInTech

Anastasi's latest video talks about this subject, though I am not linking it directly as she asks the alpaca ai to do something from Breaking Bad, which it did, and due to the legalities of such talk.. Viewer discretion is advised if you look up her vid on that.

STANFORD PAPER:

Stanford CRFM

crfm.stanford.edu