711
Dictionary

Contextual Compression

Contextual compression is a technique used to reduce the size of the input context for a Large Language Model (LLM) while retaining its most relevant information, typically by summarizing or filtering.

2 min readupdated 2026-08-04

/ quick answer

A set of techniques applied to raw input context (e.g., retrieved documents, conversation history) to reduce its token count before feeding it to a Large Language Model, typically involving summarization, filtering, or selective extraction of the most relevant information. Contextual compression is a technique used to reduce the size of the input context for a Large Language Model…

Large Language Models have finite context windows, and providing too much irrelevant or verbose information can lead to degraded performance, increased latency, and higher costs. Contextual compression addresses this challenge by intelligently reducing the volume of text fed to the LLM. This process isn't just about truncation; it involves methods like extractive summarization (selecting key sentences) or abstractive summarization (generating new, concise summaries) to preserve meaning and relevance. It can also include filtering out redundant or low-relevance information identified through semantic similarity or keyword matching. Effective contextual compression is vital for maintaining the efficiency and accuracy of LLM applications, especially in long-running conversations or when dealing with extensive knowledge bases.
Definition
A set of techniques applied to raw input context (e.g., retrieved documents, conversation history) to reduce its token count before feeding it to a Large Language Model, typically involving summarization, filtering, or selective extraction of the most relevant information.
Example
Instead of passing a full 20-page document found via RAG to the LLM, contextual compression might use a mini-LLM or a summarization algorithm to condense it into a 200-word summary, or extract only the three most semantically relevant paragraphs, before sending it along with the user query to the main LLM.
Related Workflows

/ frequently asked

What's the difference between compression and chunking?

Chunking is the process of breaking down a large document into smaller, predefined segments (chunks) for easier storage and retrieval. Compression, on the other hand, actively reduces the content within those chunks or retrieved segments by summarizing or filtering, aiming to maintain core information while reducing token count.

Can contextual compression lead to loss of important details?

Yes, there's always a trade-off. Over-compression can lead to the loss of nuanced or specific details. The key is to balance reduction in token count with the preservation of critical information. Advanced compression methods often prioritize information based on its relevance to the current user query or task.

/ topics#context-engineering#llm#summarization#token-management

/ continue exploring

Related concepts

The vocabulary this page depends on.

  • →Chunking

    Splitting documents into retrievable pieces before embedding them for RAG.

  • →Summarization

    Compressing text while preserving meaning and structure.

  • →Context Window

    The maximum amount of text (in tokens) an LLM can consider in a single call.

  • →Reranker

    A second-stage model that reorders retrieved chunks for precision.

all dictionary →

Related workflows

Turn this into a repeatable process.

  • →Context Window Optimization Workflow

    This workflow outlines steps to optimize the information fed into an LLM's finite context window, ensuring maximal relevance and efficiency while managing token limits.

  • →Run an LLM Locally

    Set up a private, offline language model on your own computer in under an hour.

  • →Dynamic Context Insertion Workflow

    This workflow details how to dynamically inject context-specific information into LLM prompts based on user queries or application state, improving response accuracy and relevance.

  • →Multi-Turn Context Management Workflow

    This workflow manages conversation history and other dynamic context in multi-turn interactions with LLMs, ensuring coherence and relevance over extended dialogues.

all workflows →

Related tool stacks

The tools that run it in production.

  • →Local AI Stack

    Open-source tools to run chat, retrieval and agents entirely on your own hardware.

  • →LLM Context Management Stack

    A technology stack for effectively managing and optimizing the context provided to large language models, ensuring efficient, relevant, and cost-effective operations.

  • →RAG Context Enrichment Stack

    A technical stack designed to enrich the contextual data provided to a Retrieval Augmented Generation (RAG) system, improving the quality and depth of LLM responses.

all tool stacks →

Comparisons & alternatives

Pick between the options.

all comparisons →

Long-form guides on this topic