The generative AI primer

At its core, a has a surprisingly straightforward job: given a sequence of text, it predicts what comes next, generating one piece at a time. Every modern concept built around these systems—autonomous agents, retrieval-augmented generation, persistent memory, tool calling, protocols like MCP, and API gateways—is scaffolding engineered around that core prediction loop to turn raw text completion into practical software.

Much of the confusion in applied AI stems from conflating the model itself with the systems built around it. Most industry vocabulary does not describe neural network weights; it describes what gets packaged into the prompt, what external actions the model can trigger, how traffic gets routed across vendors, and where the compute actually executes. Confusing these layers leads to expensive architectural mistakes—such as treating a larger context window as an alternative to a searchable knowledge base, or assuming an LLM router and an MCP gateway solve the same problem. Understanding where each layer begins and ends is essential once engineering budgets and production reliability are on the line.

This primer focuses on foundational concepts rather than vendor branding. At the bottom of most pages, vendor-specific tabs show how providers name these concepts and where their implementations diverge. Because the ecosystem evolves quickly, every claim in those tabs is explicitly marked confirmed or unconfirmed, and unverified details are noted rather than guessed.

What is in scope

This guide covers generative language models—text generation, conversational reasoning, and the surrounding plumbing that connects models to data and tools. It does not cover classical machine learning: training custom classification models from scratch, manual feature engineering, or statistical predictive modeling belong to separate disciplines. The focus here begins with pre-trained foundation models that you access through a hosted API or deploy onto rented cloud hardware.

How generation actually works

Before untangling the surrounding infrastructure, it helps to understand the underlying prediction cycle.

When you send text to a model, the text is split into —short character fragments averaging three or four letters in English. The model passes these tokens through its parameters and calculates a probability distribution across its entire vocabulary for the next potential token. A sampling algorithm selects one of those candidates, appends it to the prompt, and repeats the forward pass. This loop continues until the model emits a designated end-of-sequence token or hits an enforced length ceiling.

Two practical realities follow directly from this process:

First, the model operates purely on the input placed immediately before it. It possesses no innate recollection of previous conversations, no implicit visibility into local files, and no background awareness of the world beyond what is explicitly included in the prompt. Everything the model appears to remember must be actively re-assembled into its context window on every call.

Second, token selection incorporates controlled randomness. Sending the identical prompt multiple times can yield slightly different completions. This stochastic behavior gives language models their conversational flexibility, but it also means deterministic unit tests will break unless your architecture accounts for variance.

When a model generates fluent, convincing text that is completely untruthful, it is not broken; it is simply predicting likely token sequences without an internal fact-checking engine. The industry refers to this as a , reflecting the reality that language models lack any native way to assess whether their statements correspond to external facts.

How this primer is organised

The primer is structured into six sections, starting with the model itself and moving outward into surrounding systems. You begin with the mechanics of a single model call, explore what fits into the prompt, examine how models invoke external tools, and look at the harnesses that orchestrate multi-step tasks. The final sections cover cross-cutting organizational concerns: the gateways that manage enterprise traffic and the tradeoffs between hosted cloud APIs and self-hosted open models.

Organizing the material by architectural layer rather than by product keeps the guide durable. Specific developer tools and vendor interfaces change every quarter, but the core boundaries between contexts, harnesses, tools, and gateways remain consistent.

The model is the small box in the middle. Almost everything a practitioner designs and operates sits in the components surrounding it.

Section	Core question	Starting point
The model	How do models process text, sample outputs, and perform reasoning steps?	Tokens and the context window
Context	What can the model see, and how do you manage information within window limits?	What fills a context window
Tools and extensions	How do models interact with external systems and trigger real actions?	Tool calls
Harnesses	How does client software manage iterative execution, permissions, and state?	The agentic loop
Gateways	How do you route requests, manage credentials, and monitor usage across teams?	LLM gateways and routers
Sourcing a model	What are the tradeoffs between commercial hosted APIs and self-hosted models?	Open and closed models

Page	Core focus
Tokens and the context window	The fundamental units models read, compute, and bill on
Inference and sampling	Why completions vary and how sampling parameters influence output
Reasoning models	How test-time compute and internal reasoning chains improve complex problem solving
What fills a context window	Everything packaged alongside the user prompt and where token budgets go
Context management and compaction	How to sustain long-running sessions across strict context ceilings
Embeddings, vector stores, and RAG	How to retrieve relevant knowledge from large document collections on demand
Memory	How information persists across independent sessions
Tool calls	How models request external actions and how callers execute them
Skills, plugins, and MCP	The three primary ways to extend agent harnesses
MCP in depth	How the Model Context Protocol structures client-server communication
The agentic loop	How harnesses chain model outputs and tool execution to solve multi-step tasks
Comparing harnesses	Architectural differences across modern agent environments
LLM gateways and routers	Centralized routing, rate limiting, and observability for model calls
MCP gateways	Managing security, access control, and credentials across multiple MCP servers
Open and closed models	The operational, licensing, and legal distinctions between hosted and open weights
Local and cloud inference	Infrastructure costs, latency, hardware requirements, and deployment tradeoffs

Each page ends with the terms it introduced. The central glossary indexes all of them with direct references to their defining chapters, while the interactive quiz helps verify your practical understanding of the material.

Terms introduced

• Generative AI: systems that produce new text, images, audio, or code from a prompt, rather than classifying or scoring something that already exists.
• Large language model: a model trained to predict the next chunk of text, big enough that predicting text well requires it to encode a great deal about the world.
• Foundation model: a model trained once, at great expense, on a broad corpus, then adapted or prompted for many tasks rather than trained per task.
• Hallucination: a fluent, confident output that is not true. The model has no mechanism for telling the difference.
• Prompt: the text sent to a model for it to continue, including everything the harness added that you did not type.

---

How providers do it

The same call in three vendors' words. Two of these differences are structural rather than cosmetic: where the system prompt goes, and what the model's own turns are called.

Concept	Anthropic	OpenAI	Google
The call	POST /v1/messages	POST /responses	models.generateContent
System prompt	Top-level system; no system role	instructions, or a developer message	systemInstruction
The model's own turn	Role assistant	Output items	Role model
Conversation carried by	The full messages array, resent	The input, or a previous response id	The full contents array, resent
Sampling controls	temperature, top_p, top_k, all deprecated on the newest models	Not on the create-response reference page	generationConfig
Open weights published	None	Yes, separately from the hosted family	Yes, under the Gemma name

Every row is marked in the tab it came from. The OpenAI rows are the weakest, because its reference pages did not answer the system-prompt and sampling questions directly. This table asks nothing about model names, context window sizes or prices, and no tab on this page carries them. Where a page does ask, such as tokens and the context window, those figures are confirmed against the vendor's own documentation on the date in each tab's _Last verified line. They are also the figures that go stale fastest, so a tab that has not been re-read is a tab whose numbers should be checked before anybody quotes them.

The row that costs the most to get wrong is the first one. Three different call shapes is why an LLM gateway exists at all, and why the compatibility layer a gateway offers is never quite lossless.

Anthropic

What this maps to: Anthropic sells inference against the Claude family and does not publish weights. The unit you call is the Messages API, and the vocabulary in it lines up closely with this primer: messages, tokens, tools, a top-level system prompt.

Concept	At Anthropic	Status
The text-generation call	POST /v1/messages. Required: model, messages, max_tokens	confirmed
System prompt	A top-level system parameter. There is no system role for input messages	confirmed
Message roles	user and assistant	confirmed
Sampling controls	temperature, top_p and top_k are documented as deprecated: models released after Claude Opus 4.6 do not accept them	confirmed
Model names and context window sizes	Changes with each release	unconfirmed; read the model comparison page before quoting a number
Open weights	None published	unconfirmed; no announcement read today, and the absence of one is not a commitment

Their vocabulary

Standard term	Their term
System prompt	System prompt, passed as system
Tool definition	tools, with an input_schema
Tool call	A tool_use content block
Tool result	A tool_result content block, sent in a user message
Reasoning	Extended thinking

Where to look

The API reference for the Messages endpoint is the authoritative page for parameter names. The model comparison table is where context windows and maximum output lengths live, and it is the page that changes most often.

Last verified: 2026-09-09 against https://platform.claude.com/docs/en/api/messages.

OpenAI

What this maps to: OpenAI sells inference against the GPT family and has published open-weights models separately. It has two text-generation surfaces rather than one, which is the first thing that catches a reader out: the older Chat Completions API and the newer Responses API.

Concept	At OpenAI	Status
The text-generation call	POST /responses on the Responses API. Chat Completions remains available at POST /chat/completions	confirmed for the Responses path; the Chat Completions path is unconfirmed, not read today
Which surface to write against	The reference documents a migration guide from Chat Completions to Responses	unconfirmed; read the migration guide before starting new work
System prompt	An instructions parameter, described as a system or developer message inserted into the model's context	unconfirmed; found in the API reference index rather than on the create-response page
Message roles	user, system and developer, with developer and system taking precedence over user	unconfirmed; read from reference index text rather than the parameter table
Required parameters	The create-response reference does not mark any body parameter as required	confirmed as an observation about the page, which is itself the useful fact
Sampling controls	Not documented on the create-response reference page read today	unconfirmed
Model names and context window sizes	Changes with each release	unconfirmed; read the models page before quoting a number

Their vocabulary

Standard term	Their term
System prompt	Instructions, or a message with the developer role
Prompt	Input
Tool definition	tools, with a parameters JSON Schema
Tool call	A function call item in the output
Reasoning	Reasoning, with an effort setting

Where to look

The API reference is the authoritative page for parameter names, and the migration guide is the one that says which of the two surfaces a new application should use. One open question worth closing on any engagement: whether instructions survives across turns when conversation state is carried by a previous response id. The reference index says it does not, and that is a behaviour worth confirming directly.

Last verified: 2026-09-09 against https://developers.openai.com/api/reference/resources/responses/methods/create.

Google

What this maps to: Google sells inference against the Gemini family through the Gemini API, and separately publishes open-weights models under the Gemma name. A reader meeting both for the first time should not assume the two share a code path or a licence.

Concept	At Google	Status
The text-generation call	models.generateContent, at POST https://generativelanguage.googleapis.com/v1beta/{model=models/*}:generateContent	confirmed
Required field	contents[] is the only mandatory body field	confirmed
System prompt	A systemInstruction field in the request body, documented as text only	confirmed
Message roles	user and model. The model's own turns are model rather than assistant	confirmed
Sampling controls	temperature, topP, topK, plus candidateCount, stopSequences and maxOutputTokens, all under generationConfig	confirmed
Ranges and defaults for those controls	Not read today	unconfirmed
Model names and context window sizes	Changes with each release	unconfirmed; read the models page before quoting a number

Their vocabulary

Standard term	Their term
Message	Content, in contents[]
Assistant turn	A content with role model
System prompt	System instruction
Sampling parameters	generationConfig
Open-weights model	Gemma, as distinct from Gemini

Where to look

The generate-content API reference is the authoritative page for field names. There are two front doors to the same models, the Gemini API and Vertex AI, and the request shapes are not identical, so check which one a code sample is written against.

Last verified: 2026-09-09 against https://ai.google.dev/api/generate-content.

---

Check your understanding

0 of 4 answered

1. What operation is a large language model actually performing?

   Producing a probability for every token that could come next, then appending one and running again.Searching a database of documents it was trained on and returning the closest match.Executing a plan it wrote before it started answering.Compressing the prompt into a summary and expanding it into an answer.
2. A model states a court case that does not exist, with a plausible citation. What has gone wrong?

   Nothing has malfunctioned. The model produces likely-looking text and has no mechanism for knowing what it does not know.The model's training data contained that fake case.The model retrieved the wrong document.The temperature was set too low.
3. A team has two systems. One scores each incoming support ticket for urgency on a scale of one to five. The other drafts a reply. Which is generative AI?

   The one that drafts a reply, because it produces new text rather than assigning a label to text that already exists.Both, because both use a model.The one that scores urgency, because it makes a prediction.Neither, until a human reviews the output.
4. Why does this primer cut the subject into layers rather than into products?

   The products change every quarter and the layers do not, so a page about what a harness is outlives a page about one harness.Vendors ask for their products not to be named in training material.The layers map one to one onto the products anyway.Product comparisons cannot be kept vendor-neutral.