Test yourself

Answer before you check. The explanation matters most on the ones you get wrong, so read it when that happens.

Mixed set

0 of 10 answered

1. A generator produces bland, repetitive copy. Someone raises temperature from 0.7 to 1.4 and leaves top-p at 0.9. What does that combination do?

   It flattens the distribution so lower-ranked tokens win more often, while top-p still discards the tail below the 0.9 cumulative line.It has no effect, because top-p overrides temperature entirely.It widens the pool of eligible candidates, because temperature sets how many tokens top-p keeps.It makes the model generate longer output.
2. A harness offers `get_user` against the CRM and `lookup_customer` against the auth database. The model keeps calling the wrong one. What is the cause?

   The names and descriptions are the only thing the model has to choose with, and neither one says which system it reaches.Two tools with similar names cannot both be registered in one request.The model calls tools in the order they were defined.The model needs to try one before it can tell them apart.
3. On a typical coding-assistant call, which part of the context did the user write?

   The question, which is usually a fraction of a percent of the tokens sent.The system prompt and the question.The tool definitions, since the user chose which tools to connect.All of it. The harness only forwards what was typed.
4. A colleague says a 20,000-word document is "about 27,000 tokens". What is wrong with stating it that way?

   A token count depends on which model's tokenizer does the counting, so the figure is only true for one model family.Tokens are always whole words, so the count should equal the word count.Token counts are only defined for the model's output, not for input.Nothing is wrong. One token is four characters by definition.
5. Turning on streaming makes a chat interface feel much faster. What actually changed?

   Nothing about total generation time. Tokens are sent as they are decoded, so the reader starts reading during decode instead of after it.The model decodes tokens in parallel once streaming is enabled.Prefill is skipped, so the first token arrives sooner.The provider allocates the request to faster hardware.
6. A summarisation job sends 80,000 tokens and asks for a 200-token answer. Users report a long wait before anything appears, then a fast finish. Where did the time go?

   In prefill, which processes the whole prompt before the first output token can exist.In decode, because a long prompt makes each output token slower to generate.In the tokenizer, which is the slow step for large inputs.In network transfer of the prompt.
7. A gateway's primary model starts erroring and its fallback chain moves traffic to a weaker second model. Users report nothing. What is the risk?

   The calls succeed and return 200, so nothing signals that answer quality has dropped unless the response names the model that answered.The calls fail, so error rates rise and the applications have to handle new status codes.The fallback model rejects requests written for the primary model's API.The gateway stops counting tokens once it leaves the primary model.
8. Why does a harness run a large repository search in a subagent rather than in the main conversation?

   The subagent has its own context window, so the 60,000 tokens of file contents stay there and only the answer comes back.A subagent can call tools the main loop is not permitted to call.Subagents run on a cheaper model, which is the saving.The main loop cannot run two tools at once.
9. A test sets temperature to 0 against a hosted API and asserts on the exact response string. It passes locally and fails once a week in CI. What is the most likely cause?

   Hosted inference is not bit-reproducible, because batching, kernels and expert routing change the arithmetic slightly between runs.Temperature 0 is not respected by the API and some randomness remains in the sampler.CI sends a subtly different prompt.The test needs a longer timeout.
10. A provider returns a summary of the model's reasoning rather than the raw working. How should a debugger treat it?

    As a report about the reasoning, not a log of it, since a different model wrote the summary and a chain of thought is not a faithful record anyway.As an exact transcript, shortened for readability.As proof of which steps the model performed, useful for an audit trail.As irrelevant, since summaries carry no information at all.

Every explanation names the page that teaches the answer. If a question uses a term you have not met yet, the glossary has it.

By section

The model

0 of 8 answered

1. A generator produces bland, repetitive copy. Someone raises temperature from 0.7 to 1.4 and leaves top-p at 0.9. What does that combination do?

   It flattens the distribution so lower-ranked tokens win more often, while top-p still discards the tail below the 0.9 cumulative line.It has no effect, because top-p overrides temperature entirely.It widens the pool of eligible candidates, because temperature sets how many tokens top-p keeps.It makes the model generate longer output.
2. A retrieval step can fill a one-million-token window with everything remotely relevant, and the request still fits. Why is that a poor default?

   You pay for every token sent, prefill latency grows with prompt length, and models get worse at finding a fact buried in a very long prompt.Providers throttle any request above a few hundred thousand tokens.The model silently truncates anything past the first 100,000 tokens.A large window forces the provider to fall back to a smaller model.
3. A team pins a seed and still sees answers change after a quiet weekend with no deploys. What should they check first?

   Whether they called a moving alias rather than an exact model version, so the provider swapped the build underneath them.Whether the seed was too small to be effective.Whether temperature was left above zero, which overrides the seed.Whether the prompt contained a timestamp.
4. A reasoning model returns a 300-token answer to a 900-token prompt, having thought for 4,200 tokens first. What is billed?

   900 input tokens and 4,500 output tokens, because the thinking is generated the same way the answer is.900 input and 300 output. Thinking is internal and not billed.900 input, 300 output, and the thinking at a discounted rate.5,100 input tokens and 300 output tokens.
5. A test sets temperature to 0 against a hosted API and asserts on the exact response string. It passes locally and fails once a week in CI. What is the most likely cause?

   Hosted inference is not bit-reproducible, because batching, kernels and expert routing change the arithmetic slightly between runs.Temperature 0 is not respected by the API and some randomness remains in the sampler.CI sends a subtly different prompt.The test needs a longer timeout.
6. How does a reasoning model differ from prompting an ordinary model with "think step by step"?

   The reasoning model was trained to produce and check its working, so it backtracks when a step contradicts an earlier one.Only the reasoning model can produce intermediate steps at all.The reasoning model runs the same weights with a longer maximum output length.The reasoning model calls an external solver for the arithmetic.
7. A summarisation job sends 80,000 tokens and asks for a 200-token answer. Users report a long wait before anything appears, then a fast finish. Where did the time go?

   In prefill, which processes the whole prompt before the first output token can exist.In decode, because a long prompt makes each output token slower to generate.In the tokenizer, which is the slow step for large inputs.In network transfer of the prompt.
8. A prompt that fits comfortably in the window when it holds English documentation blows past it when the same volume of minified JSON is pasted in. Why?

   Punctuation-dense text tokenizes far less efficiently, so the same character count costs many more tokens.JSON is sent as raw bytes rather than tokens, so it bypasses the tokenizer.The model reserves extra window space whenever it detects structured data.JSON must be re-tokenized on every turn while prose is cached.

Context

0 of 8 answered

1. An agent gives a wrong answer about a file it was asked to change. What is the first thing to check?

   Whether the file's contents were ever in the context, rather than just its name.Whether the temperature is set too high.Whether the question was phrased ambiguously.Whether a larger model would do better.
2. Which change is most likely to invalidate a cached prefix for every request a service makes?

   Adding one tool to the tool definitions, which sit at the front of the request.Appending the user's new question at the end of the messages.A longer reply from the model on the previous turn.Retrieving a different set of documents for this question.
3. What produces the embeddings a vector store holds?

   A separate, smaller embedding model, run over each chunk before it is stored.The same model that answers the question, on its first call.The vector store, which computes them from the text it is given.The tokenizer, as a byproduct of splitting text into tokens.
4. Turn 20 of a conversation costs about six times what turn 3 cost, for a question of the same length. Why?

   The whole conversation history is resent on every call, so each turn pays for everything before it again.Providers raise the per-token rate as a session gets longer.The model does more work per token once it has more context to consider.The output got longer, and output tokens cost more than input tokens.
5. A long session has 140,000 tokens of history. A requirement stated at turn 6 is being ignored. Where should it be moved?

   Into the system prompt, or into the most recent message.Roughly halfway through the context, so the model reads it twice.Nowhere. If it is in the window at all, the model weights it equally.Into a tool description, so it is loaded before the conversation.
6. What does truncation lose that compaction does not?

   Any record at all of the dropped turns. Compaction leaves a summary behind; truncation leaves nothing.The system prompt, which truncation removes first.The most recent turns, which truncation removes first.Nothing. The two are different names for the same operation.
7. An agent must keep a plan across a two-hour session that will compact several times. Where should the plan live?

   In a file on disk, with a short pointer to it in the context.In the first user message, so it is the oldest thing in the history.Repeated in full at the end of every reply the model gives.In the system prompt, rewritten each time the plan changes.
8. Raising top-k from 3 to 50 makes answers worse rather than better. What is the usual fix?

   Keep retrieving 50, then rerank with a second model and pass only the best 3 to the answering model.Lower top-k to 1, so only the single nearest chunk is used.Increase the chunk size so fewer chunks are needed.Switch to a model with a larger context window and pass all 50.

Tools and extensions

0 of 8 answered

1. A model emits three tool-call blocks in one reply and the caller runs only the first, sending back one result. What happens?

   The next request is malformed, because every tool call in a turn needs a matching result before the conversation can continue.The model reruns the other two automatically on the next turn.The provider fills in empty results for the missing calls.Nothing. Unanswered calls are ignored.
2. Five AI applications each need to reach eight internal systems. What does the protocol change?

   Forty bespoke integrations become thirteen, namely eight servers and five applications that speak the protocol.It removes the need for credentials, since the protocol carries authorization itself.It lets the model connect to the systems directly.It reduces the token cost of the tool definitions.
3. What is the honest cost of installing a plugin from a public marketplace?

   It can carry hooks that execute on your machine and server configuration that hands out credentials, so it is a trust decision like any dependency install.Its skills sit in the context window whether or not they are used.It has to be reinstalled on every session start.It overrides the harness's own permission model.
4. A team is building a server that reads from an internal API using a shared service credential, for forty engineers. Which transport?

   Streamable HTTP, so one deployment holds the credential and every user reaches it over the network with their own authorization.stdio, so each engineer runs their own copy with the credential in an environment variable.stdio, because it needs no port and cannot be reached from outside.Either. The transport does not affect where credentials live.
5. A harness offers `get_user` against the CRM and `lookup_customer` against the auth database. The model keeps calling the wrong one. What is the cause?

   The names and descriptions are the only thing the model has to choose with, and neither one says which system it reaches.Two tools with similar names cannot both be registered in one request.The model calls tools in the order they were defined.The model needs to try one before it can tell them apart.
6. A team wants the model to query their read replica for order history. Skill, plugin, or MCP server?

   An MCP server, because it needs a database driver, a connection string, and a credential.A skill containing the connection string and the SQL to run.A plugin, because it is a new capability.A skill, because the schema is knowledge the model is missing.
7. Which of these is something only a plugin gives you?

   One versioned install command that gives eight teammates the same skills, hooks, and server configuration.The ability to call a tool the model could not call before.A way to keep instructions out of the context until they are needed.A guarantee that a script runs after every file edit.
8. A tool times out against the reporting database. What should the caller send back to the model?

   A readable error as the tool result, saying what failed and what to do about it, marked as an error.An exception thrown out of the loop, so the caller can retry the whole request.An empty result, so the model treats the tool as having found nothing.The raw stack trace, so the model has the most information available.

Harnesses

0 of 6 answered

1. Why does a harness run a large repository search in a subagent rather than in the main conversation?

   The subagent has its own context window, so the 60,000 tokens of file contents stay there and only the answer comes back.A subagent can call tools the main loop is not permitted to call.Subagents run on a cheaper model, which is the saving.The main loop cannot run two tools at once.
2. A model "calls a tool". Which program actually opens the file?

   The harness. The model only emits a request naming the tool and its arguments.The model, using a filesystem connection the provider gives it.The MCP server, which the model connects to directly.Whichever process the provider runs the weights on.
3. A harness runs in the terminal. What does that surface make cheap, and what does it make awkward?

   It gets your shell, your credentials and piping into other tools for free, and makes reviewing a 400 line diff awkward.It makes reviewing diffs inline cheap, and running it from a scheduled job awkward.It makes working on a repository you never cloned cheap, and reaching a local credential awkward.It survives a closed laptop, and makes correcting it while it runs awkward.
4. Work is a nightly pass over failing tests on a repository nobody will be watching. Which surface fits, and what has to be true of the permissions?

   A background or cloud runner, or a headless invocation in CI, with an unattended mode that still keeps a sandbox around tool execution.An editor extension, because diffs from the run need reviewing in the morning.A terminal program left running overnight in an interactive session.A web surface, because it needs no local install.
5. On turn three the model writes a misspelled function name into a file. By turn twelve, three more files call it by that name. What went wrong, and what stops it?

   The model read its own output back as fact. Verification the model does not control, such as running the tests or the type checker, is what contradicts it.The context window filled and compaction dropped the correct spelling. A larger window fixes it.The sampling temperature was too high. Setting it to zero fixes it.The permission model allowed edits without confirming them. Confirming each edit fixes it.
6. What does a sandbox give you that a permission allowlist does not?

   Enforcement below the level of the command text, so a script that opens files itself is still fenced in.A record of every tool call for later review.A guarantee the model will not attempt the action.Lower token usage, because fewer tool definitions are sent.

Gateways

0 of 6 answered

1. An organisation runs an LLM gateway. Its security team asks which tools its agents called last week and with whose credentials. Where does that come from?

   Not from the LLM gateway. It sits on the path to the models and never sees a tool being executed. That question belongs to an MCP gateway.From the LLM gateway's traces, which record every tool call the models made.From the model vendor, which logs tool executions against the API key.From the LLM gateway's spend report, since tool calls are billed separately.
2. Two engineers say the gateway should "route by cost". One means per request, the other means per task class. Which objection applies to the per-request version?

   The gateway only sees the prompt, so it cannot tell at request time whether a cheaper model can handle this particular one.Vendors forbid comparing their prices programmatically.Per-request routing breaks streaming, because the model must be chosen before the connection opens.Token prices change too often for a gateway to hold them.
3. A gateway's primary model starts erroring and its fallback chain moves traffic to a weaker second model. Users report nothing. What is the risk?

   The calls succeed and return 200, so nothing signals that answer quality has dropped unless the response names the model that answered.The calls fail, so error rates rise and the applications have to handle new status codes.The fallback model rejects requests written for the primary model's API.The gateway stops counting tokens once it leaves the primary model.
4. A team wants a hard monthly budget on one application's model spend. Why does the cap have to be at the gateway rather than in the vendor console?

   The gateway is on the request path, so it is the only component that can decline the call. A billing alert reports the overrun after the money is spent.Vendors do not offer any spend reporting, so the gateway is the only place the spend is visible.The gateway can renegotiate the price once the budget is close to exhausted.A vendor cap applies to the whole organisation and cannot be changed.
5. A gateway fans fifty servers into one endpoint and presents every tool to every client. What has it failed to fix?

   The tool definitions still arrive in the context window on every request, so a few hundred tools can crowd out the work before the user has typed anything.The client still has to store fifty separate configuration entries.The client can no longer tell which server a tool belongs to.Each server still needs its own credential on the developer's machine.
6. Three servers behind a gateway each expose a tool called `search`. The gateway namespaces them. What should you watch for afterwards?

   The new names are text the model reads, so a long generated prefix costs tokens on every request and can change how the model picks.Namespacing breaks the protocol, because tool names must match the server's declaration exactly.The gateway can no longer route the call, since the original name is lost.Clients have to be reconfigured with the new names by hand.

Sourcing a model

0 of 6 answered

1. A team plans to save money by unloading its local model when nobody is using it and loading it again on demand. What goes wrong?

   Loading tens of gigabytes into GPU memory takes tens of seconds, and the model answers nothing until it finishes.The weights are corrupted if they are unloaded mid-session.The inference server loses the quantization settings and reloads at full precision.The model has to be re-downloaded from the publisher each time.
2. A 70B model quantized to 4-bit and a 13B model at full 16-bit precision take about the same GPU memory. How should a team choose between them?

   Run both against an evaluation set built from the team's own task, because the trade differs by workload.Take the 70B, since more parameters always beats fewer.Take the 13B, since quantization always ruins a model.Take whichever scores higher on a published benchmark.
3. You are about to ship a fine-tuned version of a published model in a customer-facing product. Which obligation is most likely to be sitting in the licence and missed?

   A requirement to display a credit and to carry the original model's name at the front of your model's name.A requirement to publish your fine-tuning dataset.A requirement to route inference through the publisher's API.A requirement to pay a royalty per token generated.
4. A team moves off a hosted API to an open-weights model, mainly to escape vendor lock-in. What does it usually find it has taken on instead?

   A dependence on the GPUs it bought, which constrain which model it can run next.A per-token bill that turns out to be higher than the vendor's.A licence that forbids running the model in production.An obligation to publish any fine-tuned model it produces.
5. An overnight job classifies two million documents. Which throughput figure should the team tune for, and what is the trade?

   Aggregate tokens per second, accepting that each individual request gets slower as batches get bigger.Per-request tokens per second, because that is what the machine can do in total.Time to first token, because the job is latency-sensitive.Neither; batch jobs are limited by disk rather than by the model.
6. A team compares the vendor's price per million tokens against the electricity cost of its own GPUs and concludes local is ten times cheaper. What is wrong with the comparison?

   It leaves out the hardware amortised over its useful life, the staff who keep the serving stack alive, and the quality gap against the hosted model.Electricity is billed per kilowatt hour rather than per token, so the two cannot be compared at all.The vendor's price per token includes a discount the team would not qualify for.Local inference produces more tokens for the same answer, so the token counts are not comparable.

By topic

Start here

Start here

0 of 5 answered

1. What operation is a large language model actually performing?

   Producing a probability for every token that could come next, then appending one and running again.Searching a database of documents it was trained on and returning the closest match.Executing a plan it wrote before it started answering.Compressing the prompt into a summary and expanding it into an answer.
2. A model states a court case that does not exist, with a plausible citation. What has gone wrong?

   Nothing has malfunctioned. The model produces likely-looking text and has no mechanism for knowing what it does not know.The model's training data contained that fake case.The model retrieved the wrong document.The temperature was set too low.
3. A team has two systems. One scores each incoming support ticket for urgency on a scale of one to five. The other drafts a reply. Which is generative AI?

   The one that drafts a reply, because it produces new text rather than assigning a label to text that already exists.Both, because both use a model.The one that scores urgency, because it makes a prediction.Neither, until a human reviews the output.
4. Why does this primer cut the subject into layers rather than into products?

   The products change every quarter and the layers do not, so a page about what a harness is outlives a page about one harness.Vendors ask for their products not to be named in training material.The layers map one to one onto the products anyway.Product comparisons cannot be kept vendor-neutral.
5. Which of these is out of scope for this primer?

   Training a model on your own labelled dataset to predict customer churn.Working out why the same prompt gave two different answers.Deciding whether to put a document in the context window or retrieve part of it.Choosing between a hosted API and running open weights on your own hardware.

Tokens and the context window

Tokens and the context window

0 of 5 answered

1. A colleague says a 20,000-word document is "about 27,000 tokens". What is wrong with stating it that way?

   A token count depends on which model's tokenizer does the counting, so the figure is only true for one model family.Tokens are always whole words, so the count should equal the word count.Token counts are only defined for the model's output, not for input.Nothing is wrong. One token is four characters by definition.
2. A prompt that fits comfortably in the window when it holds English documentation blows past it when the same volume of minified JSON is pasted in. Why?

   Punctuation-dense text tokenizes far less efficiently, so the same character count costs many more tokens.JSON is sent as raw bytes rather than tokens, so it bypasses the tokenizer.The model reserves extra window space whenever it detects structured data.JSON must be re-tokenized on every turn while prose is cached.
3. A retrieval step can fill a one-million-token window with everything remotely relevant, and the request still fits. Why is that a poor default?

   You pay for every token sent, prefill latency grows with prompt length, and models get worse at finding a fact buried in a very long prompt.Providers throttle any request above a few hundred thousand tokens.The model silently truncates anything past the first 100,000 tokens.A large window forces the provider to fall back to a smaller model.
4. A model has a 200,000-token context window. A request carries 199,000 input tokens. What should you expect?

   There is almost no room left for the answer, because the request and the answer share one budget.A full-length answer, because the window limits input only.The provider extends the window automatically to fit the output.The oldest messages are dropped by the API so the answer fits.
5. A support bot answers in about 60 tokens per reply. The monthly bill is dominated by input tokens even though users type one short line each. What explains it?

   The system prompt, the tool definitions and the whole conversation history are resent on every request.Input tokens are billed at a higher rate than output tokens.The tokenizer runs on the provider's side and is billed per call.Short replies are billed at a minimum charge that exceeds their token count.

Inference and sampling

Inference and sampling

0 of 6 answered

1. A summarisation job sends 80,000 tokens and asks for a 200-token answer. Users report a long wait before anything appears, then a fast finish. Where did the time go?

   In prefill, which processes the whole prompt before the first output token can exist.In decode, because a long prompt makes each output token slower to generate.In the tokenizer, which is the slow step for large inputs.In network transfer of the prompt.
2. What does one forward pass through a model's weights actually produce?

   A probability for every token in the vocabulary, from which something else then picks one.The next word, chosen by the model.The complete answer, which is then split into tokens for billing.A ranked shortlist of the ten most likely continuations.
3. A test sets temperature to 0 against a hosted API and asserts on the exact response string. It passes locally and fails once a week in CI. What is the most likely cause?

   Hosted inference is not bit-reproducible, because batching, kernels and expert routing change the arithmetic slightly between runs.Temperature 0 is not respected by the API and some randomness remains in the sampler.CI sends a subtly different prompt.The test needs a longer timeout.
4. A team pins a seed and still sees answers change after a quiet weekend with no deploys. What should they check first?

   Whether they called a moving alias rather than an exact model version, so the provider swapped the build underneath them.Whether the seed was too small to be effective.Whether temperature was left above zero, which overrides the seed.Whether the prompt contained a timestamp.
5. A generator produces bland, repetitive copy. Someone raises temperature from 0.7 to 1.4 and leaves top-p at 0.9. What does that combination do?

   It flattens the distribution so lower-ranked tokens win more often, while top-p still discards the tail below the 0.9 cumulative line.It has no effect, because top-p overrides temperature entirely.It widens the pool of eligible candidates, because temperature sets how many tokens top-p keeps.It makes the model generate longer output.
6. Turning on streaming makes a chat interface feel much faster. What actually changed?

   Nothing about total generation time. Tokens are sent as they are decoded, so the reader starts reading during decode instead of after it.The model decodes tokens in parallel once streaming is enabled.Prefill is skipped, so the first token arrives sooner.The provider allocates the request to faster hardware.

Reasoning models

Reasoning models

0 of 5 answered

1. A reasoning model returns a 300-token answer to a 900-token prompt, having thought for 4,200 tokens first. What is billed?

   900 input tokens and 4,500 output tokens, because the thinking is generated the same way the answer is.900 input and 300 output. Thinking is internal and not billed.900 input, 300 output, and the thinking at a discounted rate.5,100 input tokens and 300 output tokens.
2. How does a reasoning model differ from prompting an ordinary model with "think step by step"?

   The reasoning model was trained to produce and check its working, so it backtracks when a step contradicts an earlier one.Only the reasoning model can produce intermediate steps at all.The reasoning model runs the same weights with a longer maximum output length.The reasoning model calls an external solver for the arithmetic.
3. A reasoning agent handles its first tool call correctly, then loses the thread on the second. The harness rebuilds the message list from its own store before each request. What is the likely fault?

   The harness is not replaying the thinking blocks the API returned, unmodified, alongside the tool results.The reasoning effort is set too low for a multi-step task.The tool result is too large for the context window.The model cannot call more than one tool per session.
4. A document extraction pipeline pulls three fields from an invoice, one call per document, at high volume. Reasoning effort is set to high. What is the sensible change?

   Turn reasoning down or off, because the answer is already in the prompt and thinking about it does not make it more present.Raise effort further, since accuracy on extraction scales with test-time compute.Keep the effort and enable streaming to hide the latency.Switch to a larger model at the same effort setting.
5. A provider returns a summary of the model's reasoning rather than the raw working. How should a debugger treat it?

   As a report about the reasoning, not a log of it, since a different model wrote the summary and a chain of thought is not a faithful record anyway.As an exact transcript, shortened for readability.As proof of which steps the model performed, useful for an audit trail.As irrelevant, since summaries carry no information at all.

What fills a context window

What fills a context window

0 of 5 answered

1. On a typical coding-assistant call, which part of the context did the user write?

   The question, which is usually a fraction of a percent of the tokens sent.The system prompt and the question.The tool definitions, since the user chose which tools to connect.All of it. The harness only forwards what was typed.
2. A harness has twelve MCP servers connected, each exposing about twenty tools. The user asks a question that needs no tool at all. What does that cost?

   Every tool definition is still sent and still billed, because the model has to see a tool described before it can decide not to use it.Nothing. Definitions are only sent when the model calls a tool.Nothing, because the provider caches tool definitions permanently after the first call.A small fixed fee per connected server, independent of how many tools it exposes.
3. An agent gives a wrong answer about a file it was asked to change. What is the first thing to check?

   Whether the file's contents were ever in the context, rather than just its name.Whether the temperature is set too high.Whether the question was phrased ambiguously.Whether a larger model would do better.
4. A long session has 140,000 tokens of history. A requirement stated at turn 6 is being ignored. Where should it be moved?

   Into the system prompt, or into the most recent message.Roughly halfway through the context, so the model reads it twice.Nowhere. If it is in the window at all, the model weights it equally.Into a tool description, so it is loaded before the conversation.
5. Turn 20 of a conversation costs about six times what turn 3 cost, for a question of the same length. Why?

   The whole conversation history is resent on every call, so each turn pays for everything before it again.Providers raise the per-token rate as a session gets longer.The model does more work per token once it has more context to consider.The output got longer, and output tokens cost more than input tokens.

Context management and compaction

Context management and compaction

0 of 5 answered

1. What does truncation lose that compaction does not?

   Any record at all of the dropped turns. Compaction leaves a summary behind; truncation leaves nothing.The system prompt, which truncation removes first.The most recent turns, which truncation removes first.Nothing. The two are different names for the same operation.
2. An agent finishes a task, compacts, and then starts reading a file it already read and reasoned about. What happened?

   The summary did not record that conclusion, so from the agent's point of view the work was never done.Compaction cleared the model's weights, so it forgot how to read files.The file changed, so the earlier read was invalidated.The agent hit its output limit and restarted the turn.
3. An agent must keep a plan across a two-hour session that will compact several times. Where should the plan live?

   In a file on disk, with a short pointer to it in the context.In the first user message, so it is the oldest thing in the history.Repeated in full at the end of every reply the model gives.In the system prompt, rewritten each time the plan changes.
4. A team adds the current time to the top of its system prompt so the model knows the date. The bill goes up sharply. Why?

   A cache is matched on an exact prefix from position zero, so a value that changes every call invalidates everything after it.Timestamps are billed at a higher rate than ordinary text.The extra tokens for a timestamp are large enough to notice.Caching is disabled on any request that contains a date.
5. Which change is most likely to invalidate a cached prefix for every request a service makes?

   Adding one tool to the tool definitions, which sit at the front of the request.Appending the user's new question at the end of the messages.A longer reply from the model on the previous turn.Retrieving a different set of documents for this question.

Embeddings, vector stores, and RAG

Embeddings, vector stores, and RAG

0 of 5 answered

1. What produces the embeddings a vector store holds?

   A separate, smaller embedding model, run over each chunk before it is stored.The same model that answers the question, on its first call.The vector store, which computes them from the text it is given.The tokenizer, as a byproduct of splitting text into tokens.
2. A support search built on pure vector similarity fails on queries containing product codes like AX-4471-B. Why, and what fixes it?

   An embedding turns an identifier into an approximate region of meaning, so the exact string is lost. Hybrid search adds keyword matching alongside it.The codes were never embedded. Re-embedding the corpus fixes it.The chunk size is too large. Smaller chunks make identifiers easier to match.Vector stores cannot index text containing digits or hyphens.
3. A corpus of 40 internal documents totals 90,000 tokens, every user may read all of them, and questions often require comparing two documents. Retrieval or a big window?

   Put them all in the window and cache the prefix. Retrieval adds machinery and would break the comparisons.Retrieval, because a knowledge base is always the more scalable design.Retrieval, because 90,000 tokens will not fit in any current window.Neither. A corpus this size has to be fine-tuned into the model.
4. A refunds policy states a 5 day rule in one paragraph and an exception for marketplace orders in the next. Chunking splits between them with no overlap. What is the result?

   Retrieval returns a chunk that is confidently wrong for marketplace customers, and nothing in the window contradicts it.The exception is retrieved instead, because it is the more specific text.Both chunks are always retrieved together, because they came from one document.The model notices the missing context and asks a clarifying question.
5. Raising top-k from 3 to 50 makes answers worse rather than better. What is the usual fix?

   Keep retrieving 50, then rerank with a second model and pass only the best 3 to the answering model.Lower top-k to 1, so only the single nearest chunk is used.Increase the chunk size so fewer chunks are needed.Switch to a model with a larger context window and pass all 50.

Memory

Memory

0 of 5 answered

1. A chat product greets a returning user by name. What made that possible?

   Something outside the model read the name from a store and put it in the text sent with the question.The model updated its weights during the previous conversation.The provider keeps the previous conversation loaded on the server between visits.The model recognises the user from the pattern of their writing.
2. Why can turn 12 of a conversation refer to something said at turn 3?

   Turns 1 to 11 are resent in full as part of turn 12's request.The model keeps a short-term buffer of the current session.The provider stores the session and looks up earlier turns as needed.A memory write happened automatically at turn 3.
3. Which of these belongs in persistent memory?

   The build command for this project, which has not changed in a year.The contents of the config file the agent read this morning.The API token the user pasted in to unblock a test.The list of files changed in the task that just finished.
4. What most clearly distinguishes memory from retrieval?

   Memory is a small store the system wrote about this user or this work, and it is read back unconditionally at the start of a session.Memory is stored inside the model, and retrieval is stored outside it.Memory uses embeddings and retrieval uses keyword search.Memory is written by the user and retrieval is written by the model.
5. A user says "use tabs" once about a single file. The agent writes "the user prefers tabs" to persistent memory. What goes wrong?

   A narrow instruction becomes a standing fact, arriving first in every later session and applied everywhere.The memory is too small to be useful and will be ignored.Nothing, as long as the entry is dated.The model will overwrite it the next time the user formats a file.

Tool calls

Tool calls

0 of 7 answered

1. A database tool returns a 200KB JSON blob for one lookup. Why is that a problem?

   The result is appended to the conversation, so roughly 50,000 tokens of context go on one lookup and get resent on every later turn.The API rejects tool results over 64KB.The model cannot parse JSON that large.Large results make the model more likely to hallucinate the schema.
2. A harness offers `get_user` against the CRM and `lookup_customer` against the auth database. The model keeps calling the wrong one. What is the cause?

   The names and descriptions are the only thing the model has to choose with, and neither one says which system it reaches.Two tools with similar names cannot both be registered in one request.The model calls tools in the order they were defined.The model needs to try one before it can tell them apart.
3. A model replies with a tool call for `delete_branch`. What has happened to the branch at that moment?

   Nothing. The model emitted a block naming the tool and its arguments, and the caller decides whether to run it.It has been deleted. Emitting the call is what performs the action.It has been queued for deletion by the provider.It has been deleted only if the tool was marked as having side effects.
4. One vendor's documentation says "function calling" and another says "tool use". How much does the difference matter?

   They are two names for the same mechanism, so the concept carries over and only the field names change.Function calling runs the code for you; tool use only describes what to run.Function calling is limited to one call per turn; tool use allows several.Function calling returns typed values; tool use returns text.
5. A model emits three tool-call blocks in one reply and the caller runs only the first, sending back one result. What happens?

   The next request is malformed, because every tool call in a turn needs a matching result before the conversation can continue.The model reruns the other two automatically on the next turn.The provider fills in empty results for the missing calls.Nothing. Unanswered calls are ignored.
6. A team needs a model to pull five fields out of an invoice PDF and write them into a database. Tool call or structured output?

   Structured output. The model only has to produce a reply matching a schema, and the team's own code does the writing.A tool call, because the database write is an action.A tool call, because structured output cannot enforce a JSON Schema.Neither. Extraction needs a fine-tuned model.
7. A tool times out against the reporting database. What should the caller send back to the model?

   A readable error as the tool result, saying what failed and what to do about it, marked as an error.An exception thrown out of the loop, so the caller can retry the whole request.An empty result, so the model treats the tool as having found nothing.The raw stack trace, so the model has the most information available.

Skills, plugins, and MCP

Skills, plugins, and MCP

0 of 6 answered

1. Why does installing a hundred skills cost almost nothing until they are used?

   Only each skill's name and description stay in the context. The body is read in only when a request matches.Skills are compressed and expanded by the harness on demand.Skills are stored on the provider's side and never enter the context.The harness picks the ten most recently used skills and drops the rest.
2. A team writes a skill describing how to file a ticket in their internal tracker. The model still cannot file tickets. Why?

   A skill is instructions. It changes how the model uses capabilities it already has, and the model has no way to reach the tracker.The skill's description was not specific enough to trigger.Skills only work when installed as part of a plugin.The skill body was too long to load into the context.
3. A team wants the model to query their read replica for order history. Skill, plugin, or MCP server?

   An MCP server, because it needs a database driver, a connection string, and a credential.A skill containing the connection string and the SQL to run.A plugin, because it is a new capability.A skill, because the schema is knowledge the model is missing.
4. A procedure should run only when a developer explicitly asks for it, never on the model's own judgement. Skill or slash command?

   A slash command, because the user invokes it by name, where a skill is loaded when the model decides it is relevant.A skill, because skills are always loaded on demand.Either, since both are invoked the same way.Neither. An explicit procedure needs an MCP server.
5. Which of these is something only a plugin gives you?

   One versioned install command that gives eight teammates the same skills, hooks, and server configuration.The ability to call a tool the model could not call before.A way to keep instructions out of the context until they are needed.A guarantee that a script runs after every file edit.
6. What is the honest cost of installing a plugin from a public marketplace?

   It can carry hooks that execute on your machine and server configuration that hands out credentials, so it is a trust decision like any dependency install.Its skills sit in the context window whether or not they are used.It has to be reinstalled on every session start.It overrides the harness's own permission model.

MCP in depth

MCP in depth

0 of 6 answered

1. Five AI applications each need to reach eight internal systems. What does the protocol change?

   Forty bespoke integrations become thirteen, namely eight servers and five applications that speak the protocol.It removes the need for credentials, since the protocol carries authorization itself.It lets the model connect to the systems directly.It reduces the token cost of the tool definitions.
2. An editor is configured with three MCP servers. How many MCP clients are involved?

   Three. A client speaks the protocol to exactly one server, and the host holds one client per server.One. The client is the editor, and it connects to all three servers.Four. One client per server plus one for the host itself.It depends on the transport. stdio servers share a client.
3. A server exposes a database schema as a resource. Why does the model not fetch it whenever it wants?

   Resources are application-controlled. The host decides what to attach, usually because the user picked it.Resources are only readable during the initial handshake.Resources cannot be represented as text in the context.The model has to call a tool to convert the resource first.
4. Which cost of a connected MCP server is invisible in a demo with one server?

   Every server's tool definitions occupy the context window on every call, and tool names from different servers collide.The handshake adds latency to the first request.The server has to be restarted between conversations.Resources are re-read on every turn.
5. A team is building a server that reads from an internal API using a shared service credential, for forty engineers. Which transport?

   Streamable HTTP, so one deployment holds the credential and every user reaches it over the network with their own authorization.stdio, so each engineer runs their own copy with the credential in an environment variable.stdio, because it needs no port and cannot be reached from outside.Either. The transport does not affect where credentials live.
6. A client written against the 2025-11-25 revision cannot connect to a server built for 2026-07-28. What changed?

   The newer revision dropped the `initialize` handshake for a stateless model where every request carries its version and capabilities.The newer revision dropped stdio in favour of HTTP only.Tool definitions moved from JSON Schema to a proprietary format.Nothing changed. Protocol versions are advisory.

The agentic loop

The agentic loop

0 of 6 answered

1. A model "calls a tool". Which program actually opens the file?

   The harness. The model only emits a request naming the tool and its arguments.The model, using a filesystem connection the provider gives it.The MCP server, which the model connects to directly.Whichever process the provider runs the weights on.
2. What ends a normal pass through the agentic loop?

   The model replies with text and asks for no tool.The harness runs out of tools to offer.The model reaches the end of its context window.The user sends a second message.
3. Why does a harness run a large repository search in a subagent rather than in the main conversation?

   The subagent has its own context window, so the 60,000 tokens of file contents stay there and only the answer comes back.A subagent can call tools the main loop is not permitted to call.Subagents run on a cheaper model, which is the saving.The main loop cannot run two tools at once.
4. What does a sandbox give you that a permission allowlist does not?

   Enforcement below the level of the command text, so a script that opens files itself is still fenced in.A record of every tool call for later review.A guarantee the model will not attempt the action.Lower token usage, because fewer tool definitions are sent.
5. An overnight run burned a large budget and produced nothing. The transcript shows the same tool failing 300 times with slightly different arguments. What would have caught this?

   A turn limit, or a rule stopping the run after repeated failures of the same tool.A larger context window, so the model could see all the earlier failures.A stricter permission model, so the tool needed confirmation.Automatic compaction, so the failures were summarised away.
6. On turn three the model writes a misspelled function name into a file. By turn twelve, three more files call it by that name. What went wrong, and what stops it?

   The model read its own output back as fact. Verification the model does not control, such as running the tests or the type checker, is what contradicts it.The context window filled and compaction dropped the correct spelling. A larger window fixes it.The sampling temperature was too high. Setting it to zero fixes it.The permission model allowed edits without confirming them. Confirming each edit fixes it.

Comparing harnesses

Comparing harnesses

0 of 6 answered

1. A team is told to switch from one harness to another next quarter. Which of their work carries over unchanged?

   The agent instructions file in the repository root.The permission allowlist.The skills they wrote.The hooks that run their formatter after every edit.
2. An organisation has bought an LLM gateway to cap spend per team and log every request. Which property of a harness decides whether the gateway can cover it?

   Whether the harness lets you set the endpoint it calls.Whether the harness supports MCP.Whether the harness has a headless mode.Whether the harness reads an instructions file from the repository.
3. A harness advertises checkpointing. What should you check before relying on it to undo a long run?

   Whether it captures changes made by shell commands and by subagents, or only edits made through its own file-editing tools.Whether checkpoints are stored in the same directory as the transcript.Whether the checkpoint includes the token counts for each turn.Whether the harness compacts the conversation before checkpointing.
4. A harness runs in the terminal. What does that surface make cheap, and what does it make awkward?

   It gets your shell, your credentials and piping into other tools for free, and makes reviewing a 400 line diff awkward.It makes reviewing diffs inline cheap, and running it from a scheduled job awkward.It makes working on a repository you never cloned cheap, and reaching a local credential awkward.It survives a closed laptop, and makes correcting it while it runs awkward.
5. Why is headless mode the axis that decides whether a harness can do automated pull request review?

   A pipeline has no interactive session, so the harness must take one prompt, run to completion, and set an exit code.Pipelines cannot run a harness that uses MCP servers.Headless mode is the only mode that compacts the context automatically.Interactive harnesses cannot read a repository they did not clone themselves.
6. Work is a nightly pass over failing tests on a repository nobody will be watching. Which surface fits, and what has to be true of the permissions?

   A background or cloud runner, or a headless invocation in CI, with an unattended mode that still keeps a sandbox around tool execution.An editor extension, because diffs from the run need reviewing in the morning.A terminal program left running overnight in an interactive session.A web surface, because it needs no local install.

LLM gateways and routers

LLM gateways and routers

0 of 7 answered

1. A gateway's primary model starts erroring and its fallback chain moves traffic to a weaker second model. Users report nothing. What is the risk?

   The calls succeed and return 200, so nothing signals that answer quality has dropped unless the response names the model that answered.The calls fail, so error rates rise and the applications have to handle new status codes.The fallback model rejects requests written for the primary model's API.The gateway stops counting tokens once it leaves the primary model.
2. A team wants a hard monthly budget on one application's model spend. Why does the cap have to be at the gateway rather than in the vendor console?

   The gateway is on the request path, so it is the only component that can decline the call. A billing alert reports the overrun after the money is spent.Vendors do not offer any spend reporting, so the gateway is the only place the spend is visible.The gateway can renegotiate the price once the budget is close to exhausted.A vendor cap applies to the whole organisation and cannot be changed.
3. Four teams each hold their own vendor API key. What does putting an LLM gateway in front of them change first?

   The applications hold gateway-issued keys, the gateway holds the vendor credentials, and every call can be attributed to a team.The vendor starts invoicing per team instead of per organisation.The models get cheaper, because the gateway negotiates a volume rate.The applications stop needing to handle errors, because the gateway retries everything.
4. One team's batch job exhausts the organisation's whole quota with a vendor every morning, and three other teams get errors. What does a gateway do about it?

   Rate limits and queues per team, so the batch job waits inside its own share instead of consuming everybody's.Routes the batch job to a different vendor, which is the only available fix.Caches the batch job's requests so they stop reaching the vendor.Raises the vendor's limit automatically.
5. An organisation runs an LLM gateway. Its security team asks which tools its agents called last week and with whose credentials. Where does that come from?

   Not from the LLM gateway. It sits on the path to the models and never sees a tool being executed. That question belongs to an MCP gateway.From the LLM gateway's traces, which record every tool call the models made.From the model vendor, which logs tool executions against the API key.From the LLM gateway's spend report, since tool calls are billed separately.
6. A gateway's semantic cache is set with a loose distance threshold. Which failure should you expect?

   A question that differs from a cached one only by a negation, a date or a name gets the cached answer, and neither the caller nor the user can tell.The cache stops hitting, because no two prompts are ever close enough.Cached answers become stale and the gateway returns an error.The cache returns a partial response that the application has to reassemble.
7. Two engineers say the gateway should "route by cost". One means per request, the other means per task class. Which objection applies to the per-request version?

   The gateway only sees the prompt, so it cannot tell at request time whether a cheaper model can handle this particular one.Vendors forbid comparing their prices programmatically.Per-request routing breaks streaming, because the model must be chosen before the connection opens.Token prices change too often for a gateway to hold them.

MCP gateways

MCP gateways

0 of 7 answered

1. A row goes missing from the production database. Why can no individual MCP server answer "which person's agent did this"?

   A server sees a connection and a request. Which human is behind the client does not survive the hop, so only the gateway knows both ends.MCP servers are forbidden from logging requests by the protocol.The model, not the client, issues the call, so there is no human to name.Database servers log at the table level and cannot attribute a row.
2. A gateway offers policy either per server or per tool. Why is per-server granularity usually not enough?

   One server commonly exposes both harmless read tools and destructive write tools, so allowing the server allows both.Server names change between releases, so a per-server rule stops matching.A server can be reached directly, bypassing any per-server rule.Per-server rules cannot be expressed for remote servers, only for local ones.
3. Forty developers each have fifty MCP servers configured locally. Which problem is the main reason to put a gateway in front of them?

   Every developer holds a credential for every system, on a laptop, and nobody can revoke or rotate them centrally.Fifty local server processes use too much memory on a developer machine.MCP servers cannot be reached from more than one client at a time.The protocol has a limit on how many servers one client may connect to.
4. Why does a server registry change the risk of running MCP servers, when the gateway already enforces policy on every call?

   Policy controls what an approved server may be used for. The registry decides what gets approved at all, with an owner and a pinned version against each entry.The registry replaces policy enforcement, so only one of the two is needed.A registry removes the need to hold credentials centrally.Registered servers do not consume context window space.
5. A finance director wants model spend broken down by team. The organisation already runs an MCP gateway. What does that give them?

   Nothing. An MCP gateway sits on the path to tools and data and never sees a prompt or a completion. Spend and model choice belong to an LLM gateway.A per-team cost report, since the MCP gateway authenticates every user.Token counts per request, because tool results pass through it.A model routing table it can price up.
6. A gateway fans fifty servers into one endpoint and presents every tool to every client. What has it failed to fix?

   The tool definitions still arrive in the context window on every request, so a few hundred tools can crowd out the work before the user has typed anything.The client still has to store fifty separate configuration entries.The client can no longer tell which server a tool belongs to.Each server still needs its own credential on the developer's machine.
7. Three servers behind a gateway each expose a tool called `search`. The gateway namespaces them. What should you watch for afterwards?

   The new names are text the model reads, so a long generated prefix costs tokens on every request and can change how the model picks.Namespacing breaks the protocol, because tool names must match the server's declaration exactly.The gateway can no longer route the call, since the original name is lost.Clients have to be reconfigured with the new names by hand.

Open and closed models

Open and closed models

0 of 6 answered

1. A vendor's download page calls its model "open source". The licence is named after the model, incorporates an acceptable use policy, and requires a separate agreement above 700 million monthly active users. Is the model open source?

   No. An open source licence cannot restrict fields of use or add a user threshold.Yes. The weights are downloadable and can be fine-tuned, which is what open source means for a model.Yes, as long as your own product stays below the threshold.It depends on whether the training code was released alongside the weights.
2. A publisher releases the weights for a model on a download site. What has it released?

   A file of numbers produced by the training run, which needs separate code to run.The program that runs the model, with the numbers compiled into it.The training data the model learned from.A licence to call the publisher's hosted API without a key.
3. A team builds a product on a closed model reached only through the vendor's API. What risk does that carry which an open-weights model does not?

   The vendor can change or retire the model behind the name the team calls, on its own schedule.The team cannot control how many tokens the model produces.The prompts the team writes will not transfer to any other model.The team has no way to measure how the model behaves on its own tasks.
4. You are about to ship a fine-tuned version of a published model in a customer-facing product. Which obligation is most likely to be sitting in the licence and missed?

   A requirement to display a credit and to carry the original model's name at the front of your model's name.A requirement to publish your fine-tuning dataset.A requirement to route inference through the publisher's API.A requirement to pay a royalty per token generated.
5. Why are published weights described as closer to a compiled binary than to source code?

   You can run them and modify them by fine-tuning, but you cannot read how they came to be or rebuild them.They are compressed, so they have to be decompiled before use.They contain machine instructions rather than numbers.They only run on the hardware the publisher compiled them for.
6. A team moves off a hosted API to an open-weights model, mainly to escape vendor lock-in. What does it usually find it has taken on instead?

   A dependence on the GPUs it bought, which constrain which model it can run next.A per-token bill that turns out to be higher than the vendor's.A licence that forbids running the model in production.An obligation to publish any fine-tuned model it produces.

Local and cloud inference

Local and cloud inference

0 of 6 answered

1. A 70B model quantized to 4-bit and a 13B model at full 16-bit precision take about the same GPU memory. How should a team choose between them?

   Run both against an evaluation set built from the team's own task, because the trade differs by workload.Take the 70B, since more parameters always beats fewer.Take the 13B, since quantization always ruins a model.Take whichever scores higher on a published benchmark.
2. What is the first calculation to do before running a model on your own hardware?

   Parameters times bytes per parameter, to see whether the weights fit in GPU memory.Requests per second divided by tokens per second.The size of the context window in tokens.The vendor's price per million tokens against your electricity cost.
3. A team plans to save money by unloading its local model when nobody is using it and loading it again on demand. What goes wrong?

   Loading tens of gigabytes into GPU memory takes tens of seconds, and the model answers nothing until it finishes.The weights are corrupted if they are unloaded mid-session.The inference server loses the quantization settings and reloads at full precision.The model has to be re-downloaded from the publisher each time.
4. A team compares the vendor's price per million tokens against the electricity cost of its own GPUs and concludes local is ten times cheaper. What is wrong with the comparison?

   It leaves out the hardware amortised over its useful life, the staff who keep the serving stack alive, and the quality gap against the hosted model.Electricity is billed per kilowatt hour rather than per token, so the two cannot be compared at all.The vendor's price per token includes a discount the team would not qualify for.Local inference produces more tokens for the same answer, so the token counts are not comparable.
5. A single user runs a model on an idle GPU at home. Why is the cost per token so poor?

   Generating a token moves the whole weights file through the compute units, and with one request that pass serves one user instead of thirty.A local inference server cannot use the GPU's full clock speed.The electricity cost of an idle GPU is higher than the cost of a busy one.Quantized models are slower per token than full-precision ones.
6. An overnight job classifies two million documents. Which throughput figure should the team tune for, and what is the trade?

   Aggregate tokens per second, accepting that each individual request gets slower as batches get bigger.Per-request tokens per second, because that is what the machine can do in total.Time to first token, because the job is latency-sensitive.Neither; batch jobs are limited by disk rather than by the model.