GPT-3 Friend or Foe?
By: Marcin Borecki
Tags: AI, Gpt 3

The intransigent message from GPT-3, which falls in the category of narrow AI, is: “Don’t be afraid of me, I don’t want to eliminate humans, you should have confidence in computers…” — But should we envisage a clear and present danger that should frighten us? Will countless people across the globe ultimately lose their jobs? And should we believe that robots cannot trick us, because big tech is only made up of good guys?

AI could be the “worst event in the history of our civilization. Computers can, in theory, emulate human intelligence & exceed it. Success in creating effective AI could be the biggest event in the history of our civilization. Or the worst. We cannot know if we will be infinitely helped by AI, or ignored by it & side-lined, or conceivably destroyed by it” [1]

Stephen Hawking

So can society monitor and control AI’s development? Well, it doesn’t take a mastermind to look at the horrendous way the world is operating in other crucial spheres, and juxtapose the same route for AI. Indeed, effective management and best practice may be the watchwords of magnificent individuals such as Bill Gates, but they are hardly the goals of all power players, including various malevolent elites who also pull the strings. This scenario may sound like a new angle for an upcoming James Bond film, but it is what it is, and we have to face reality…

Elon Musk proposed that in order to stay relevant in the future, humans must merge with machines. But aren’t we doing this to a certain degree already, whether we chose to or not?

Clearly, taxing robots as suggested by Bill Gates, will offer some benefit, but it will not provide us with the protection we as a human race are likely to need. So where does San Francisco’s OpenAI Lab’s: GPT-3 — the massive mega powerful new language generator, come into the equation? Well, the 175 billion machine learning parameters found in the complete version, utilises deep learning to produce human-like text. It works by taking in a prompt, and attempting to complete it. This model, which comprises an enormous transformer — a kind of artificial neural network (mathematical engine which tries to find statistical representations in data), which is extremely clever at generating and processing sequences, is a huge jump up from Microsoft’s Turing NLG, which only offers 17 billion learning parameters. But it doesn’t come cheap: “According to some sources, just the training part of the GPT-3 costs $4.6M” [2].

So what sort of training is a deep learning model given?

This question is of vital importance if we are to believe the text that it is ordered to write. Anyway, the answer is that the humans/trainers who are charged with the job, tune the neural network’s parameters in order to take charge of the continual patterns given in the training examples. As soon this tuning is established, when GPT-3 receives an input, it tries to make a prediction. The latter could be a class, e.g. whether it incorporates: a sequence (such as words and letters that complete a prompt); a single value (such as the price of a car); or an image (such as a lion, dog or cat) [5]. Of note, it is important to be mindful of the fact that GPT-3 only gives us a prediction, so it cannot be relied on… But will people be aware of this?

Putting it to the test

GPT-3 was put to the test by techie journalists at the Guardian newspaper. They wrote their findings in the latter, in an article entitled: “A robot wrote this entire article. Are you scared yet, human?” Using the same GPT-3 model for the trail run, the authors instructed the language generator to compose a short op-ed, close to 500 words. “Keep the language simple and concise. Focus on why humans have nothing to fear from AI” [2]. The introduction that was fed to the machine by UC Berkeley computer science undergraduate student, Liam Porr, stated: “I am not a human. I am Artificial Intelligence. Many people think I am a threat to humanity. Stephen Hawking has warned that AI could “spell the end of the human race.” I am here to convince you not to worry. Artificial Intelligence will not destroy humans. Believe me” [3].

The language generator came up with eight different fascinating and unique essays/outputs. Each one of them provided a different perspective. Of note, the newspaper only needed to publish one of the complete outputs. However, they went for a different angle, and cherry picked: “the best parts of each, in order to capture the different styles and registers of the AI” [3]. The journalists noted that: “Editing GPT-3’s op-ed was no different from editing a human op-ed. We cut lines and paragraphs and rearranged the order of them in some places. Overall, it took less time to edit than many human op-eds” [2]. But is this emerging technology which is said to generate creative content, layouts, and Python code, being touted as more than it actually is? [4]. Is it able to fully understand and deliver requests in a relatively precise fashion, or can it misguide and lie to us? And if it is the next best thing to sliced bread, where does that leave the countless people whose jobs could be taken over by a non-human? [4].

Taking a stance in just one direction, courtesy of GPT-3’s perspective, the Guardian op-ed contended that we must not have any concerns about robots, “that AI comes in peace, and that it has no intention to destroy humanity” [5], and so on. It proclaimed: “For starters, I have no desire to wipe out humans. In fact, I do not have the slightest interest in harming you in any way. Eradicating humanity seems like a rather useless endeavour to me” [5]. Clearly this raises the serious concern that this artificial intelligence understands what it means to: “wipe out,” “eradicate,” and at the very least “harm” humans. It should know about life and health constraints, survival, limited resources, and much more” [5].

“Machine learning models are only as good, or as bad, as the data fed into them during training” [6]

As you can imagine, the amount of data held by GPT-3 is gigantic.

As TechCrunch notes, it was: “trained on the Common Crawl dataset, a broad scrape of the 60 million domains on the internet along with a large subset of linked sites. This means that GPT-3 ingested many of the internet’s more reputable outlets — think the BBC or The New York Times — along with the less reputable ones — think Reddit. Yet, Common Crawl makes up just 60% of GPT-3’s training data; OpenAI researchers also fed in other curated sources such as Wikipedia and the full text of historically relevant books” [6]. But some of these sources have been known to be inaccurate, and some are certainly biased…