Question 1

What makes a language model large?

Accepted Answer

The word 'large' refers to the number of parameters — the weights that encode the model's learned knowledge. Modern frontier LLMs have hundreds of billions of parameters. Larger models generally have greater capability but cost more to train and run.

Question 2

What is hallucination in LLMs?

Accepted Answer

Hallucination is when an LLM generates text that sounds confident and fluent but is factually wrong or fabricated. It occurs because LLMs predict probable tokens rather than retrieving facts. Grounding techniques like RAG help, but hallucinations cannot be fully eliminated.

Question 3

What is the context window of an LLM?

Accepted Answer

The context window is the maximum number of tokens the model can consider at once — both the input (prompt) and the output (response). Models like GPT-4o support 128k tokens; Claude 3.5 Sonnet supports 200k tokens. Longer contexts allow processing of full books or large codebases.

Question 4

What is the difference between an open and closed LLM?

Accepted Answer

Closed models (GPT-4o, Claude, Gemini) are proprietary — you access them via API but cannot inspect or modify the weights. Open-weight models (Llama 3, Mistral, Qwen) release their weights publicly, allowing local deployment, fine-tuning, and modification.

Question 5

How are LLMs evaluated?

Accepted Answer

LLMs are evaluated on benchmarks like MMLU (knowledge), HumanEval (coding), MATH (mathematics), and MT-Bench (instruction following). Human preference evaluations (like Chatbot Arena) are increasingly important because benchmarks can be gamed through training.

LLM (Large Language Model)

What Is LLM (Large Language Model)?

Why It Matters

Real-World Examples

Common Use Cases

Frequently Asked Questions