This is a good one. The answer to this question is both very simple and very complex. Here is the simple (and very simplified) answer:

Text is converted into tokens (indices of the tokenizer's vocabulary), so the model does not work with text directly but with the indices of the tokenizer's dictionary.
The tokens are then organized into a tensor for processing. This tensor is typically a 2D tensor, where each row represents a sequence of tokens. The number of rows corresponds to the number of sequences being processed simultaneously (the batch size), and the number of columns corresponds to the length of the longest sequence in the batch (the sequence length). Each element in the tensor is an integer that represents a specific token.
The tensor is then passed through an embedding layer, which converts each token (integer) into a dense vector of fixed size. The embedding layer is essentially a lookup table, where each row corresponds to a token in the vocabulary. It is expected that this embedding vector captures the semantic meaning of the input text.
These embeddings undergo a series of arithmetic operations in each hidden layer of the model. Each element of the hidden layer matrix represents a real number, known as a weight. Weights are "learned" - approximated during training from an enormous amount of data. The set of weights learned from the data represents the probabilistic behavior of the data processed during training (please note that the weights do not represent probabilities directly). The embeddings are multiplied by these weights. This is followed by the addition of a bias term and the application of a non-linear activation function (like swish). This process transforms the embeddings, allowing the model to learn complex patterns in the data. This process continues for all the hidden layers, and this is where the "magic" happens. I can't explain this magic because I'm under-qualified for this. However, what I can tell is that probabilities of tokens appearing close to each other or far from each other are stored in the network of weights.
The final layer of the model produces a tensor of logits, which represent the model’s predictions for the next token in the sequence for each input token.
Depending on the architecture, a softmax function may be applied to these logits to produce a probability distribution over the vocabulary for each predicted token. The method for selecting the predicted token can vary, including choosing the token with the highest probability, using beam search, nucleus sampling, or other methods specified in the architecture. This is also part of the "magic".
These predicted tokens are then converted back into human-readable text, forming the model’s generated response. This is how the model processes text and generates responses.

You might find clues to the complex part in the simple answer tagged as "magic". I'm under-qualified for the magic part. You need to ask those with extensive knowledge of mathematics and probability theory to reveal the actual math behind the magic for you.

But there is a part that no one can answer at the moment: How do the relations between tokens caught from the training data cause the spatial comprehension and basic arithmetic capabilities (yet very limited) in LLMs? We can't answer this question for living creatures as well. How can a goldfish distinguish "less" from "more", and how can chickens count to 10?

In any case, LLMs are very spectacular predictors for what they have learned, and for what they haven't, they are even more spectacular.