What Is a Large Language Model (LLM)?

Large language models (LLMs) are deep learning systems trained on vast amounts of text so they can understand and generate human language. They are the technology behind chatbots such as ChatGPT, Claude and Gemini, and behind most AI writing tools used in content and marketing today.

This guide explains what LLMs are, how they are built, which ones exist today, what they can and cannot do, and how they differ from AI, generative AI and GPT.

What Are Large Language Models (LLMs)?

A large language model (LLM) is a type of artificial intelligence trained on massive amounts of text that can understand and write natural language for many different jobs, from answering questions to summarizing documents and writing code.

The word "large" refers to two things at once: the scale of the training data and the size of the model. Model size is measured in parameters, the internal numerical weights the model adjusts during training. GPT-3, described by OpenAI researchers in 2020, has 175 billion parameters. Today's frontier models are trained on far more text, but most vendors don't publish their parameter counts anymore.

LLMs are built on the transformer architecture, introduced in the 2017 research paper "Attention Is All You Need" by Vaswani et al. Its self-attention mechanism lets the model weigh how relevant every token in a passage is to every other token. That's how an LLM can follow a long document, something older language models couldn't do well.

LLMs reached a mass audience on 30 November 2022, when OpenAI released ChatGPT as a free research preview. Since then, nearly every major technology company has shipped its own model family, and it's easy to lose track of them.

How Do Large Language Models Work?

LLMs work by repeatedly predicting the next token in a sequence, and they learn grammar, facts, reasoning patterns and writing style from billions of training examples.

A token is the unit of text a model reads and writes: a whole word, a part of a word or a punctuation mark. The context window is the maximum number of tokens the model can consider at once, covering both your prompt and its answer. Anything beyond the context window is invisible to the model, so context size really matters for long documents.

Building an LLM happens in three broad stages:

1. Step 1: Pretraining on Massive Text Data

   During pretraining, the model reads a very large corpus of text and learns to predict the next token, billions of times over, with no labeled answers.

   The GPT-3 paper gives a concrete sense of scale: its Common Crawl portion started as 45TB of compressed plain text and was filtered down to 570GB, then combined with WebText2, two book corpora and English Wikipedia. By the end of pretraining the model has broad language ability but isn't yet tuned for any specific task or for following instructions.

2. Step 2: Fine-Tuning for Specific Tasks

   Fine-tuning is a supervised training process in which a pretrained model is trained further on a smaller, task-specific dataset of instructions paired with desired outputs.

   A model fine-tuned for customer support behaves differently from one fine-tuned for code generation or legal summarization, even if both started from the same pretrained base. Fine-tuning lets organizations adapt a general-purpose LLM to their own workflows without training a model from scratch.

3. Step 3: RLHF (Reinforcement Learning from Human Feedback)

   Reinforcement learning from human feedback (RLHF) is a refinement step in which human reviewers rank different model outputs and the model is trained to prefer the responses people rate higher.

   RLHF is a large part of why modern LLMs behave like conversational assistants rather than raw text-completion engines. It makes answers far more helpful, reduces harmful output and improves instruction-following without requiring new training data for every behavior.

Examples of Large Language Models

The best-known large language models come from five families: GPT (OpenAI), Claude (Anthropic), Gemini (Google), Llama (Meta) and Mistral (Mistral AI); the first three are closed models offered through apps and APIs, while Llama and Mistral publish their weights.

The table lists the current flagship or best-known model of each family with the context window the vendor states. All values were checked against the vendors' own documentation on 1 November 2026 and change often, so confirm them before you build on them.

Llama 4 Scout and Mistral Large 3 are mixture-of-experts models: only part of the network, the "active" parameters, runs for each token, which keeps inference cheaper than the total parameter count suggests, and that's the main reason the design is so popular. Meta describes Scout's 10 million tokens as its supported maximum, so practical limits depend on where and how you run the model, and you shouldn't assume the full figure is available everywhere. Mistral's documentation now lists Mistral Large 3 as deprecated in favor of a newer model, so treat it as an example of an open Apache 2.0 release, not a current recommendation.

Context windows have grown by orders of magnitude. The original ChatGPT model could handle only a few thousand tokens, roughly a few pages of text. A million-token window can easily hold several long reports or a full contract set in a single prompt.

LLMs are usually grouped along three lines: open-weight versus closed (can you download and run the model yourself), general-purpose versus domain-specific (a model tuned for code, law or medicine), and text-only versus multimodal (a model that also reads images or audio). Open-weight models like Llama and Mistral can run on your own infrastructure, which gives you real control over your data. Closed models are reached through an API, and the vendor maintains them. You can't inspect or host them yourself.

What Can Large Language Models Do?

Large language models can generate text, summarize documents, translate languages, answer questions, write and debug code and power conversational chatbots.

• Text generation: write articles, emails, product descriptions, social posts and ad copy from a prompt
• Summarization: condense long documents, reports or research papers into short outputs
• Translation: translate between many language pairs while keeping context
• Question answering: answer questions from a provided document or knowledge base
• Code generation: write, complete and debug code in languages such as Python, JavaScript and SQL
• Classification and sentiment analysis: categorize content, detect tone and label text at scale
• Conversational AI: keep context across a multi-turn conversation to power support bots, virtual assistants and AI agents

For content teams, the practical skill is clearly knowing what to hand to an LLM (first drafts, summaries, variations) and where human judgment must stay in charge (facts, positioning, final edits). If you want to understand how LLMs affect search visibility, our guide to LLM SEO covers the topic in depth.

LLM vs. AI vs. GPT: What Is the Difference?

AI is the broad field, machine learning is one way to build AI, LLMs are machine learning models that work with language, and GPT is one specific LLM series made by OpenAI. All GPT models are LLMs, but not all LLMs are GPT.

A useful analogy: if AI is the category "smartphones," LLMs are one type of smartphone and GPT is one brand within that type.

GPT stands for Generative Pre-trained Transformer. Each word describes a core property of the model: it generates text (Generative), was trained on unlabeled text before task-specific tuning (Pre-trained) and uses the transformer architecture (Transformer).

LLM vs. generative AI: generative AI is the wider category of systems that create new content, including images, audio, video and code. LLMs are the text-focused part of it. Our explainer on what generative AI means covers the wider field.

LLM vs. NLP: natural language processing (NLP) is the older, broader field of making computers work with human language. Earlier NLP relied on rules and statistical methods; LLMs apply deep learning at a scale that makes them far more flexible across open-ended tasks.

LLM vs. chatbot: a chatbot is an application and an LLM is the technology that powers it. ChatGPT is a chatbot built on OpenAI's GPT models, and Claude is Anthropic's assistant built on its Claude models. That's the whole distinction. The LLM is the engine; the chatbot is the interface.

What Are the Limitations of Large Language Models?

The main limitation of large language models is hallucination: they can state false information fluently and confidently. Beyond that, LLMs reflect bias in their training data, have a knowledge cutoff and can struggle with long multi-step reasoning.

• Hallucinations: OpenAI defines hallucinations as plausible but false statements generated by language models, and its September 2025 research argues they persist partly because training and evaluation reward guessing over admitting uncertainty. A model can very easily cite a study that does not exist or attribute a quote to the wrong person, so verify every factual claim before you publish it. Don't skip that step.
• Training data bias: LLMs inherit, quite literally, the viewpoints, demographics and languages that dominate their training text, which can shape tone, framing and representation in ways that are easy to miss without deliberate review.
• Knowledge cutoff: a model knows nothing after its training data ends, so questions about recent events or new products can return outdated or invented answers unless the system adds live information.
• Reasoning limits: LLMs handle many reasoning tasks well, but they can easily slip on multi-step math, formal logic and tasks that need exact step-by-step checking. Reasoning-focused models, such as OpenAI's o-series, reduce this with extra chain-of-thought training, but the limit hasn't disappeared.

The most common practical fix for knowledge cutoff and hallucination is retrieval-augmented generation (RAG): the system first retrieves relevant documents and hands them to the model with your question, so the answer is grounded in sources you control. RAG lowers the risk but doesn't remove it, which is why human review still matters a great deal. Our guide to AI visibility covers how search engines and AI systems judge accuracy and freshness.

Frequently Asked Questions About Large Language Models