Using only the Decoder in GPT offers a few advantages. First, it reduces the computational complexity compared to using both the Encoder and Decoder. Second, it simplifies the model architecture and makes it easier to train and optimize for language generation tasks. Finally, it allows GPT to focus on generating high-quality outputs by solely focusing on the autoregressive generation process.