This is my amateur description of how an autoregressive language model works after a few days of casual research, I'm trying to describe it to ensure that I fully understand what's happening and learn more from my errors - I'd love to know of any criticims or ways that you think this description could be improved please as that would be very helpful, I'm seeking to minimise my degree of error and I'm hoping that you might be able to help.

So ChatGPT is a large language model, which is where all of the text on the internet is taken and analysed. The analysis of it is something called training, you have thousands of graphical processing units that people use for gaming making associations between words and generating probabilities. Attached the the LLM is a transformer, the transformer, the transformer is a process with a series of stages whereby a user can input data, and it figures out how to generate the desired output. That's how people interact with ChatGPT. So the transformer has two main components, an encoder and a decoder. The encoder starts first and the data is put into a sub-component caller input embedding, where it's turned into a multi-dimensional vector, whereby it makes associations between the input words (which by this point have been processes into numbers) and other words in the LLM's lexicon that have associations with them. For example, the word king might be associated with crowns, robes, palaces etc. The semantic relationship between words is also mapped. Then the tokens go through process encoding. This is where the order of the words is analysed to make sense of them. Then they go into something called a multi-head attention subcomponent. This is where weights are added to the words. Weights are the importance that each word has in a sentence, for example, 'calculate'. It does that using a scaled product attention mechanism, so it knows the attention score of each word, i.e, how much attention to give each word due to its importance in the sentence. That's connected to something called an add + norm subcomponent. This is where the data is normalised and the input data is added back into it to keep it on course, ready for the next component, which is called a feed forward subcomponent. The feed forward is like an comet pushing through a field of asteroids, but all the while sparks are coming off it and connecting with the asteroids, which are forming the neurons of the neural net (like a brain), but at the same time moving to process new information. That's connected to another add + norm subcomponent. So that's the encoder done. That connects the the second element of the decoder. The decoder is where the output is being generated, it starts second and looks one space in the sentence to the right of the encoder, the first two steps are similar to the encoder, but this time it's output embedding. Then it goes into something called a masked-multi-head attention subcomponent, which is like a filter for things that the LLM isn't allowed to generate outputs for. Then another multi-head attention, then an add + norm, then a feed forward, an add + norm then it goes into a linear subcomponent, which is where words are rearranged to help the tranaformer make sense of them, then into a softmax subcomponent, which is like a voting system to define probabilities of all or the filler words, like 'and', 'the' etc, then the output is generated from that.