What fills a context window
When you send a single line of text to a modern coding assistant, the model rarely receives just that single line. Behind the scenes, the application packages your query inside an extensive payload that can easily exceed thirty thousand tokens—of which your prompt accounts for less than one percent. Everything else is assembled dynamically by the client software and injected into the request without ever appearing on your screen.
Context management and compaction
As an interactive session continues, the message history steadily expands until it threatens to exceed the model's context window. The underlying model cannot decide what to keep or discard when this happens. That responsibility falls entirely to the client harness orchestrating the conversation. How that software manages information at the context boundary determines whether an agent successfully finishes a long task or quietly forgets its original instructions halfway through.
Embeddings, vector stores, and RAG
Imagine managing 12,000 customer support articles totaling over 40 million tokens, while your language model operates with a 200,000-token context window. Even with modern long-context models, you cannot simply dump the entire knowledge archive into a single prompt. An external system must pinpoint the handful of articles relevant to a user's question and deliver those excerpts to the model at query time.
Memory
Large language models do not retain memory across independent calls. Between requests, model weights remain completely static: nothing you type alters internal parameters, and each new API call executes in total isolation from the last. When a conversational application greets you by name or remembers your preferences from last week, an external software layer fetched those details from a database and injected them directly into the prompt payload.