원본 캡처
verified · type: reference · [meta, raw]

I have enough to write authoritative, current definitions. Here is the compiled glossary.

LLM Concepts & Glossary — Practitioner Reference

A tight, practical foundation. Each entry: a 2–4 sentence definition plus a "why it matters in practice" line. Definitions of fast-moving concepts (reasoning models, context engineering, quantization, evals) were sanity-checked against 2025–2026 sources; links at the bottom.

Tokens & Tokenization

A token is the atomic unit an LLM actually reads and generates — not a word, but a sub-word chunk (roughly ¾ of a word in English, so ~750 words ≈ 1,000 tokens). Tokenization is the process of splitting text into these units via a fixed vocabulary; the same model always uses the same tokenizer. Rare words, code, non-English scripts, and whitespace often fragment into many tokens, inflating counts unpredictably.

Why it matters in practice: Tokens are the billing and limit unit — cost, latency, and context-window budget are all counted in tokens, not characters or words. Count tokens (don't eyeball them) before shipping long prompts, and expect non-English or code-heavy text to cost 2–3× more per "word."

Context Window

The context window is the maximum number of tokens a model can consider at once — the combined size of your system prompt, conversation history, retrieved documents, the user's input, and the model's output. Modern windows range from ~8K to 1M+ tokens. It is a hard ceiling: exceed it and the earliest content is dropped or the request errors.

Why it matters in practice: A large window is not free capacity — cost and latency scale with what you actually fill, and models degrade on information buried in the middle of very long contexts ("lost in the middle"). Treat the window as a budget to curate, not a bucket to dump into.

Temperature, Top-p & Sampling Parameters

These control randomness in token selection. Temperature (typically 0–2) flattens or sharpens the probability distribution: low (0–0.3) is near-deterministic and focused; high (0.8+) is diverse and creative. Top-p (nucleus sampling) restricts choices to the smallest set of tokens whose cumulative probability reaches p (e.g. 0.9); top-k limits to the k most likely tokens.

Why it matters in practice: Use low temperature for extraction, classification, code, and anything needing reproducibility; higher for brainstorming and copywriting. Tune temperature or top-p, not both aggressively — and note that even temperature 0 is rarely perfectly deterministic in production.

An embedding is a fixed-length numeric vector (e.g. 768 or 1,536 dimensions) that represents the meaning of a piece of text, so that semantically similar texts land near each other in vector space. Vector search stores these embeddings in a vector database and, given a query embedding, returns the nearest neighbors by cosine similarity — retrieving by meaning rather than keyword match.

Why it matters in practice: This is the retrieval engine under most RAG and semantic-search systems. Chunking strategy, the choice of embedding model, and the fact that embeddings match similarity (not correctness or recency) are the levers — and the pitfalls — that decide retrieval quality.

RAG (Retrieval-Augmented Generation)

RAG injects external knowledge into a model at answer time: the user's query retrieves relevant documents (usually via vector search), those documents are pasted into the prompt as context, and the model answers grounded in them. It gives the model access to private, current, or domain-specific data it was never trained on — without retraining. The model reads the retrieved text; it does not memorize it.

Why it matters in practice: RAG is the default fix for "the model doesn't know our data" and for reducing hallucination via grounding + citations. It shines when knowledge changes frequently (weekly or faster), where fine-tuned models go stale — but its ceiling is retrieval quality: garbage retrieved is garbage answered.

Fine-tuning vs. Prompting vs. RAG — When to Use Which

Three distinct levers, best understood by what each changes: prompting changes the instructions; RAG changes the knowledge the model can see; fine-tuning changes the model's behavior by continuing its training. The practitioner rule: start with prompting (hours), add RAG when the model needs data it was never trained on (real-time or private knowledge), and fine-tune only when prompting and RAG both fall short on consistent behavior, output format, tone, or latency.

Why it matters in practice: Teams routinely burn money fine-tuning what a better prompt or RAG would have solved — a fine-tuned pipeline's total cost of ownership can run 10–50× a well-built RAG system, and it goes stale as your data changes. Reach for fine-tuning for style/behavior, RAG for knowledge, prompting for everything first; mature production systems layer all three.

Need Reach for
Better output on a task you can describe Prompting
Answers from private / current / large knowledge RAG
Consistent tone, format, behavior, or lower latency Fine-tuning
Knowledge that changes weekly or faster RAG (never fine-tuning)

Hallucination

A hallucination is output that is fluent, confident, and wrong — fabricated facts, citations, APIs, or quotes that the model presents as true. It stems from how LLMs work: they predict plausible next tokens, not retrieve verified facts, so there is no built-in distinction between "known" and "made up." It is most dangerous precisely because the tone is indistinguishable from correct answers.

Why it matters in practice: Never ship raw LLM output as fact in high-stakes domains without grounding (RAG), citations, or verification. Mitigation is engineering — retrieval, tool-use, constrained outputs, and evals — not a setting you can turn off.

Quantization

Quantization compresses a model's weights from high precision (16-bit) down to 8-, 4-, or fewer bits, drastically cutting memory and often speeding inference, while preserving most quality. Common formats: GGUF (CPU+GPU hybrid, for llama.cpp/Ollama — Q4_K_M is the go-to size/quality balance), and AWQ/GPTQ (GPU-only serving, e.g. vLLM). 4-bit is now practical for real-world use, with quality loss often hard to spot in normal conversation.

Why it matters in practice: Quantization is what lets a large model run on a single GPU, a laptop, or half the VRAM — the difference between "can't deploy" and "runs locally." But avoid aggressive 4-bit for math, code, and reasoning-heavy work, where the quality loss shows most; pick the format that matches your serving stack.

Reasoning / Thinking Models

Reasoning models (e.g. the o-series, DeepSeek-R1, and "thinking" modes) are trained to generate an explicit intermediate chain of thought before the final answer — spending extra inference-time compute ("slow thinking") on deliberate, step-by-step problem solving instead of answering in one pass ("fast thinking"). They markedly outperform standard models on math, coding, logic, and multi-step planning. The reasoning trace is often hidden or summarized, but you pay for those thinking tokens.

Why it matters in practice: Use them for genuinely hard, multi-step problems — and not for simple lookups or formatting, where they're slower, pricier, and can overthink. The trade-off is latency and cost (thinking tokens add up) against accuracy on tasks that actually need reasoning.

Multimodality

A multimodal model accepts and/or produces more than one type of data — text plus images, audio, video, or documents — in a shared representation, rather than text alone. In practice this means you can pass a screenshot, PDF, chart, or photo directly into the prompt and ask about it, or get images/audio as output. Capabilities vary sharply by model and by direction (understanding an image vs. generating one).

Why it matters in practice: It unlocks whole workflows — document/receipt parsing, screenshot debugging, visual QA, chart reading — that previously needed separate OCR or vision pipelines. Verify exactly which modalities a given model supports as input vs. output before designing around it, and note images consume tokens too.

Context Engineering

Context engineering is the discipline of systematically assembling everything that goes into the context window — system instructions, retrieved documents, tool outputs, conversation state, and examples — so the model has exactly the right information to do the job reliably. It's the superset that prompt engineering now lives inside: prompting crafts the individual instructions; context engineering architects what fills the window and how. Karpathy called it "the delicate art and science of filling the context window with just the right information."

Why it matters in practice: For agents and production systems, most failures are context failures — missing, stale, bloated, or poorly ordered information — not bad prompts. As of mid-2025 the industry framing shifted decisively toward this view ("context engineering is in"); managing the window as a curated, dynamic system is the core skill for anything beyond a single-turn chatbot.

Evals (Evaluations)

Evals are systematic tests that measure whether an LLM system actually does its job — the equivalent of a test suite for non-deterministic software. They span: rule-based/reference checks (exact match, regex, does-it-parse), LLM-as-judge (using a strong model to score, classify, or compare outputs — pointwise scoring or pairwise "which is better"), and human review for high-stakes or ambiguous cases. Robust evaluation blends offline tests, human judgment, and production telemetry.

Why it matters in practice: Without evals you're shipping vibes — you can't tell if a prompt change, model swap, or new RAG index made things better or worse. LLM-as-judge lets you evaluate 100K+ outputs in hours at near-zero cost (vs. weeks of human review), so build a small eval set before you start tuning, and reserve human review for calibrating the judge and edge cases.

Sources (for the fast-moving definitions)