Retrieval-Augmented Generation Track
A TREC evaluation track for studying and benchmarking systems which combine retrieval and generation.
Runs are due on August 8th! You may submit up to 10 runs per task. Submit your runs through Evalbase and report a priority on each submission form, so we can prioritize those runs for manual assessments.
Latest from the track
RAGDoll evaluation toolkit
Run the full TREC RAG evaluation workflow end to end, from generating evaluation artifacts to scoring LLM responses.
Try RAGDollNew corpus: NVIDIA ClimbMix-400b
ClimbMix-400b replaces MS MARCO v2.1 for 2026. Access it now through the Pyserini REST API!
Announcement REST API SKILLA unified benchmark for retrieval-augmented generation
The TREC Retrieval-Augmented Generation Track fosters innovation and research in retrieval-augmented generation systems: combining retrieval methods that find relevant information within large corpora with Large Language Models, to help systems produce relevant, accurate, updated, and contextually appropriate content.
The track brings the research community together around a unified benchmark to evaluate the end-to-end performance of systems that combine retrieval and generation. Distinct but complementary tasks enable deeper analysis of individual components and their interactions.
Two tasks for 2026
Pick one or both. Full input/output specs live in the agent-ready track guidelines.
Retrieval
Given a narrative, return a ranked list containing all and only the ClimbMix documents that the system predicts are relevant to that narrative and useful as evidence for answer generation.
View task guidelinesRetrieval-Augmented Generation
Given a narrative, retrieve relevant evidence from the ClimbMix collection and return a summarized answer grounded in that evidence.
View task guidelinesKey dates for 2026
-
Test topics releasedJuly 6, 2026
-
Baselines releasedWeek of July 19, 2026
-
Submission deadlineAugust 8, 2026
-
Results & judgments returnedTBD
-
TREC 2026 ConferenceNovember 2026
Everything you need to participate
2026 test data
Official test topics for the 2026 track.
2026 development data
Practice topics, RAG25 nuggets, UMBRELA qrels, and ResearchRubrics for system testing.
RAGDoll
Automated end-to-end framework for evaluating TREC RAG systems, from gold-standard construction to scoring long-form answers.
ClimbMix-400b corpus
NVIDIA's ClimbMix-400b, served through the Pyserini REST API skill.
Organizing committee
Explore related TREC tracks
Consider also participating in other RAG-related tracks at TREC 2026.