From Markov Chains to Thinking Code Models
How does a system turn a few simple probability rules into one that understands text, breaks down complex problems, and even writes code?
The inner workings of modern large language models might seem like magic at first, but their foundations are built on well-traceable ideas: Markov chains, neural networks, embeddings, and the attention mechanism of the Transformer architecture. In this overview, we trace how these models learn from text, how they handle context, and why they are capable of increasingly complex reasoning.
The goal is not to hide the mathematical details, but to present them in an accessible way—making it clearer what happens behind the scenes when an LLM responds, thinks, or generates code.