Most explanations of LLMs stop at "it predicts the next word." Here is what actually happens inside the model.

  1. Tokenizer: "What is gravity?" is split into subword tokens and mapped to integer IDs.
  2. Embeddings: each ID looks up a learned vector. In Llama 3 8B, that vector has 4,096 numbers.
  3. Transformer blocks: multi-head self-attention computes softmax(QKᵀ/√d)·V, so every token weighs every earlier token. A feed-forward network (SwiGLU in Llama) transforms the result. This repeats across N layers, 32 in Llama 3 8B.
    4.