What architecture does GPT use?


GPT uses the transformer architecture which is a neural network architecture based on the attention mechanism to find global dependencies between the input and the output. To be more specific, GPT belongs to a sub type in the transformer family that is called decoder-only transformer.