Retrieval-Augmented Generation Track
A TREC (NIST) benchmark bringing the research community together to evaluate end-to-end systems that combine retrieval with large language models.
Latest from the track
RAGDoll evaluation toolkit
Run the full RAG evaluation workflow end to end — from generating evaluation artifacts to scoring LLM responses.
Try RAGDollNew corpus: NVIDIA ClimbMix-400b
ClimbMix-400b replaces MS MARCO v2.1, accessed through the Pyserini REST API.
AnnouncementA unified benchmark for retrieval-augmented generation
The TREC Retrieval-Augmented Generation Track fosters innovation and research in retrieval-augmented generation systems: combining retrieval methods that find relevant information within large corpora with Large Language Models, to help systems produce relevant, accurate, updated, and contextually appropriate content.
The track brings the research community together around a unified benchmark to evaluate the end-to-end performance of systems that combine retrieval and generation. Distinct but complementary tasks enable deeper analysis of individual components and their interactions.
Two tasks for 2026
Pick one or both. Full input/output specs live in the agent-ready track guidelines.
Retrieval
The classic IR task: given a list of topics and access to the ClimbMix collection — via the Pyserini REST API or your own system — retrieve and rank the most relevant segments.
View task guidelinesRetrieval-Augmented Generation
Retrieve relevant evidence and return a summarized answer grounded in that evidence — citing the supporting segments so results stay reproducible.
View task guidelinesKey dates for 2026
-
Test topics releasedJuly 6, 2026
-
Baselines releasedSoon after July 6
-
Submission deadlineAugust 8, 2026
-
Results & judgments returnedTBD
-
TREC 2026 ConferenceNovember 2026
Everything you need to participate
2026 test data
Official test topics for the 2026 track.
2026 development data
Practice topics, RAG25 nuggets, UMBRELA qrels, and ResearchRubrics for system testing.
RAGDoll
Automated end-to-end framework for evaluating TREC RAG systems, from gold-standard construction to scoring long-form answers.
ClimbMix-400b corpus
NVIDIA's ClimbMix-400b, served through the Pyserini REST API skill.
Organizing committee
