GPT-3 could be revolutionizing for the future AI
By: Hamza Abdullah
Tags: Artificial Intelligence, Machine Learning, Deep Learning, Technology

Artificial intelligence is currently progressing like an infant developing in early childhood. There’s a great deal of hype around moving towards real time AI. An exciting new advances in deep learning — GPT-3 a computer program developed by Elon Musk backed research driven startup “OpenAI”.

GPT-3 is so human that most of you can’t even judge whether a text is produced by a human or GPT-3. The question arise here that what is GPT-3?

In this article you will get to know the initial glimpse of new exciting advances in deep learning.

What is GPT-3?

GPT-3 or Generative Pre-trained Transformer 3 — a third version of the original piece of software.

Which produces the text using pre-trained gigantic algorithm gathered by crawling the internet they need to carry out their task. If you ask a question or want a summary then you mostly like to get a answer of that question or desired summary. It is one of the largest artificial neural network ever created. It is currently in private beta for which people can sign up on a wait-list.

It can amazingly do pretty much anything that has a language structure — it can answer questions, writing essays, summaries, language (s) translations, take memos, and even create computer code.

How does it work?

It is different than usually neural networks which on training only spit out numerical set of data only computer can understands. GPT or more specifically GPT-3 is a prediction model which generates domain specific output after training. It means that it is an algorithmic structure designed to take one piece of text (an input) and transform it into what it predicts is the most useful following piece of text for the user.

GPT-3 fed with billionth of bits of data during its training phase converting it into vector form for numerical representations then after maximum accurate correlation — converted it back to the valid sentence.

Once it has been trained, meaning, its calculations of conditional probability across billions of words are made as accurate as possible, then it can predict what words come next when it is prompted by a person typing an initial word or words.

OpenAI has invested great deal of money for computing resources needed by GPT-3 to understand how languages work and are structured.

The number of weights the GPT-3 dynamically holds in its memory and uses to process each query is 175 billion — 10 times more than any previous non-sparse language model. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning. It only requires few-shot demonstrations via textual interaction with the model.

This tremendous leap in its ability make GPT-3 exceptional in the field outperforming any previous model (s).

Limitations

Every new breakthrough has some limitations which need time, effort and capital to become completely flawless innovation. GPT-3 ability to perform plenty of text production tasks are praised by many but all comes with some important consideration.

The CEO of OpenAI himself, Sam Altman, has said, “The GPT-3 Hype is too much. AI is going to change the world, but GPT-3 is just an early glimpse."

Firstly, it is expensive. Because it need huge amount of computing resources to carry out its function.

Secondly, it is privately held not open sourced. OpenAI has not revealed any in-depth details of how its algorithms work?

Thirdly, the output of the system is still not perfect.

These are some of the issues which can be addressed over time.

GPT-3 has ignited a new spark in the development of AI. Its most striking feature is its generality. Previously neural networks were task/domain specific with specifically trained data. But GPT-3 has no task-specific functions, and it needs no special dataset. It simply takes the maximum input and produces the sensible output in return.

Further reading:

Original GPT-3 research paper